Tool: Monument Classification Algorithms (Supervised Learning)
Pencils of Promise (PoP) is an international non-profit dedicated to providing quality education to children all over the world. They work to improve and evaluate student literacy by partnering with communities to design, implement, and monitor a package of holistic programs in rural public primary schools in Ghana, Guatemala and Laos. With the approval and support of national governments, PoP builds public schools, trains public teachers via its Teacher Support (TS) programming in order to improve student literacy outcomes, and ensures a safe and healthy learning environment through Water, Sanitation, and Hygiene (WASH) infrastructure and programming.
As part of this work, PoP collects longitudinal data for measuring student performance before, during, and after an education intervention. The insights in these data are important because they can help PoP allocate resources more efficiently to improve teacher support and consequently student outcomes, evaluate and ensure equity of outcomes, and administer donor money effectively.
The Challenge
Pencil of Promise’s Teacher Support (TS) program in Ghana was part of a study to analyze how student performance across groups changes over time. PoP used a carefully validated literacy tool called the Early Grade Reading Assessment to assist in providing results. They sampled students from schools receiving Teacher Support as well as non-TS schools. PoP wanted to examine their hypothesis that students in TS schools would perform better than those who weren’t in TS schools when adjusting for a variety of structural and demographic variables. This would help validate their program improvements and inform future work. The analysis is also useful for examining equity across gender, ethnic, language, and age groups.
Pencils of Promise sampled 1,275 students across 15 Teacher Supported and 5 non-Teacher Supported schools. To analyze these results, PoP will have to lay much of the groundwork. To begin, they have to make sure their data set is in the correct formatting, both in structure and data type. Any errors in formatting will lead to incorrect data points or careful reformatting. Once that is completed, they import it into a data analytics program. After, they must build their own models and configure the layers of data. Then, they have to go through the long process of model building and evaluating the data. Next, they may have to create another algorithm to predict data points. Given the intensity of doing this manually, there is substantial risk for an analyst to overlook small but key components and collaborating on models is fraught with package dependency and other development environment issues. This process can take days to weeks to complete with traditional modeling approaches. Monument can complete all of these tasks in a manner that was not thought of before. Modeling with Monument is quick and easy.
The Solution
With Monument’s platform, Pencils of Promise was able to find insights in record time. The process consisted of two parts. First, Pencils of Promise’s data was loaded into Monument. Then, Monument ran a Light Gradient Boosted Machine (LGBM) on each of the ten subjects in the treatment test, and also against demographic features. This skipped the time-consuming process of handling data with code and tuning algorithms with code.
Monument completed the preliminary analysis and determined that the indicator for TS participation has importance, though not a statistically significant one in this analysis. It accounts for around 10% of features used for most subjects. Monument found the most important features to be the most linearly distributed test sections, which account for 40% — 50% of features used. Based on this promising analysis, PoP will proceed with using the dataset in their analysis, likely utilizing a mixed-effects model after strengthening their survey-weighting and finite population corrections.
The Benefits
Pencils of Promise quickly advanced to their next internal stage with the Programs team, cognizant the results they received were accurate and actionable. This work will help advance their research across all PoP programs.
Pencils of Promise now knows which features of the assessment play a major role in predicting the scores. In the future, they can adjust their environments accordingly, whether something needs more or less attention.
Conclusion
Pencils of Promise was able to amplify their analytical effectiveness by freeing themselves of the ordinary ways of using data. They no longer needed to go through the time consuming, and inefficient steps to obtain solutions. With the time saved, they could easily run more tests without the feeling of uneasiness and work overload.
With Monument, data analysts can jump right in and find insights from data. Monument is a no-code tool to use Machine Intelligence to its fullest effect. Gone are the days of lengthy codes and models that no one understands after the core developer has left the team.
Interested in learning more about Monument? Book a free introductory Zoom call here.
