Analysis of the impact of the stack system on learner engagement, performance, and perception on its use in mathematics at Maseno University, Kenya

Abstract

Feedback plays a crucial role in enhancing student learning outcomes. In large classes it is instrumental in maintaining student engagement, motivation, and overall academic performance by addressing misconceptions, and fostering deeper understanding of content amongst others. The School of Mathematics, Statistics, and Actuarial Science at Maseno University has had challenges with carrying out frequent formative assessments in its 20 high-enrollment introductory courses, due to large class sizes (500 to 1000 students) and limited lecturer support, affecting student engagement with content. In 2019, the faculty recognized the need for technological solutions to overcome the limitations of traditional assessment called the STACK system. The STACK system is a digital online platform with the ability to deploy grade randomized, advanced and adaptive mathematics related questions as well as provide immediate feedback, making it a valuable tool for large classes with diverse learning needs. However, despite its implementation, a research gap exists regarding its effectiveness as a solution for improving student engagement, performance, and perception in mathematics. The purpose of this study was to analyze the impact of the STACK system on learner performance, factors affecting engagement and learner perception on its use in mathematics at Maseno University. The research objectives were to: examine the correlation between learner test results within the STACK system and the scores in the end-of-semester exams, evaluate the factors affecting learner engagement with the STACK system and the end-of-semester exams and evaluate learner perceptions regarding the use of the STACK system as a formative assessment tool in mathematics. This study was guided by Gagne’s theory of 9 levels of learning. This study was done at Maseno University, SMSAS. Population of the study was 4417 students enrolled in 5 high-enrolment courses already using STACK.Purposive sampling was used to select a single course for analysis which had 517 students enrolled. The study used STACK Quiz Analysis Guide, Students' Score Card, Online Survey Questionnaire, Interviews, and Focus Group Discussions (FGDs) to collect data. Cronbach's alpha (α = 0.87) was used to check the survey questionnaire's reliability. Reliability and validity of the remaining tools were checked by experts in the field of education. Pearson's correlation analysis revealed a significant positive relationship between STACK scores and the end-of-semester exam (r=0.63, p ≤ 0.01), as well as the frequency of STACK engagement and end-of-semester exam scores (r=0.61, p ≤ 0.01). Key factors affecting learner engagement, are feedback, randomization, quiz availability, and peer interaction, having both positive and negative impacts on student learning and final outcome in the course. The online survey, interviews, and Focus Group Discussions (FGDs) revealed a generally positive perception of the use of STACK in formative assessment in mathematics. Despite this, concerns were expressed regarding insufficient feedback from certain STACK questions. These findings align with Gagne's theory of learning, emphasizing the significance of formative assessment, learner engagement, individual differences and positive perception, for effective learning. The study recommends further investigation to explore the impact of different types of feedback on learner cognition in undergraduate mathematics, to address the concerns raised by the students about insufficient feedback in certain STACK questions.