Koedinger, K. R., & Corbett, A. (2005). Cognitive Tutors: Technology Bringing Learning Sciences to the Classroom. In R. K. Sawyer (Ed.), The Cambridge Handbook of the Learning Sciences (pp. 61–78). Cambridge University Press. https://doi.org/10.1017/CBO9780511816833.006
Koedinger, K. R., D’Mello, S., McLaughlin, E. A., Pardos, Z. A., & Rosé, C. P. (2015). Data Mining and Education. WIREs Cognitive Science, 6(4), 333–353. https://doi.org/10.1002/wcs.1350
Koedinger, K. R., Stamper, J. C., McLaughlin, E. A., & Nixon, T. (2013). Using Data-Driven Discovery of Better Student Models to Improve Student Learning. In H. C. Lane, K. Yacef, J. Mostow, & P. Pavlik (Eds.), Artificial Intelligence in Education (pp. 421–430). Springer. https://doi.org/10.1007/978-3-642-39112-5_43
Kreuter, F., & Peng, R. D. (2014). Extracting Information from Big Data: Issues of Measurement, Inference and Linkage. In H. Nissenbaum, J. Lane, S. Bender, & V. Stodden (Eds.), Privacy, Big Data, and the Public Good: Frameworks for Engagement (pp. 257–275). Cambridge University Press. https://doi.org/10.1017/CBO9781107590205.016
Krumm, A. E., & Beattie, R. (2017). Strategies for Making Digital Learning System Data Usable: A Design Workshop Approach. AERA Anual Meeting, San Antonio, TX.
Krumm, A. E., Yeager, D. S., & Yamada, H. (2019). Explanatory and Predictive Modeling Within Improvement Science Projects. AERA Annual Meeting, Toronto, ON. http://tinyurl.com/yaxj9my4
This paper describes a partnership-driven approach for developing measures of learning behaviors using event data from a digital learning environment is used in all grades and subjects within a charter management organization. The approach that we developed and followed included involved (1) gathering leaders’, teachers’, and students’ perspectives on learning behaviors; (2) collaboratively analyzing data; (3) using exploratory factor analyses to generate an single score; and (4) conducting explicit model-based tests that assessed the degree to which the single score was correlated with outcomes that were important to all members of the partnership.
Krumm, A. E., Beattie, R., Takahashi, S., D’Angelo, C., Feng, M., & Cheng, B. (2016). Practical Measurement and Productive Persistence: Strategies for Using Digital Learning System Data to Drive Improvement. Journal of Learning Analytics, 3(2), Article 2. https://doi.org/10.18608/jla.2016.32.6
Krumm, A. E., Boyce, J., & Everson, H. T. (2021). A Collaborative Approach to Sharing Learner Event Data. Journal of Learning Analytics, 8(2), Article 2. https://doi.org/10.18608/jla.2021.7375
Liu, R., & Koedinger, K. R. (2017). Closing the Loop: Automated Data-Driven Cognitive Model Discoveries Lead to Improved Instruction and Learning Gains. Journal of Educational Data Mining, 9(1), Article 1. https://doi.org/10.5281/zenodo.3554625
Liu, R., Stamper, J. C., & Davenport, J. (2018). A Novel Method for the In-Depth Multimodal Analysis of Student Learning Trajectories in Intelligent Tutoring Systems. Journal of Learning Analytics, 5(1), Article 1. https://doi.org/10.18608/jla.2018.51.4
Means, B. (2018). Tinkering Toward a Learning Utopia: Implementing Learning Engineering. In C. Dede, J. Richards, & B. Saxberg (Eds.), Learning Engineering for Online Education. Routledge.
Meyer, A., Grunow, A., & Krumm, A. E. (2017). Are these Changes an Improvement? Using Measures to Inform Homework Practices. AERA Annual Meeting, San Antonio, TX.
Dede, C. (2015). Data-Intensive Research in Education: Current Work and Next Steps. Computing Research Association. https://cra.org/wp-content/uploads/2015/10/CRAEducationReport2015.pdf
Nelson, I. A., London, R. A., & Strobel, K. R. (2015). Reinventing the Role of the University Researcher. Educational Researcher, 44(1), 17–26. https://doi.org/10.3102/0013189X15570387
Paquette, L., Ocumpaugh, J., Li, Z., Andres, A., & Baker, R. (2020). Who’s Learning? Using Demographics in EDM Research. Journal of Educational Data Mining, 12(3), Article 3. https://doi.org/10.5281/zenodo.4143612
Piety, P. J. (2019). Components, Infrastructures, and Capacity: The Quest for the Impact of Actionable Data Use on P–20 Educator Practice. Review of Research in Education, 43(1), 394–421. https://doi.org/10.3102/0091732X18821116
Piety, P. J., Hickey, D. T., & Bishop, M. J. (2014). Educational data sciences: Framing emergent practices for analytics of learning, organizations, and systems. Proceedings of the Fourth International Conference on Learning Analytics And Knowledge, 193–202. https://doi.org/10.1145/2567574.2567582
Piety, P. J., & Pea, R. D. (2018). Understanding Learning Analytics Across Practices. In D. Niemi, R. D. Pea, B. Saxberg, & R. E. Clark (Eds.), Learning Analytics in Education (pp. 215–232). IAP.
Reardon, S. F. (2019). Educational Opportunity in Early and Middle Childhood: Using Full Population Administrative Data to Study Variation by Place and Age. RSF: The Russell Sage Foundation Journal of the Social Sciences, 5(2), 40–68. https://doi.org/10.7758/rsf.2019.5.2.03
Roschelle, J., Knudsen, J., & Hegedus, S. (2010). From New Technological Infrastructures to Curricular Activity Systems: Advanced Designs for Teaching and Learning. In M. J. Jacobson & P. Reimann (Eds.), Designs for Learning Environments of the Future: International Perspectives from the Learning Sciences (pp. 233–262). Springer US. https://doi.org/10.1007/978-0-387-88279-6_9
Luckin, R., Hansen, C., Wasson, B., Clark, W., Avramides, K., Hunter, J., & Oliver, M. (2015). Teacher Inquiry into Students’ Learning: Researching Pedagogical Innovations. In P. Reimann, S. Bull, M. Kickmeier-Rust, R. Vatrapu, & B. Wasson (Eds.), Measuring and Visualizing Learning in the Information-Rich Classroom. Routledge.
Scoville, R., & Little, K. (2014). Comparing Lean and Quality Improvement [White Paper]. Institute for Healthcare Improvement (IHI). https://www.ihi.org:443/resources/Pages/IHIWhitePapers/ComparingLeanandQualityImprovement.aspx
Sendak, M. P., Balu, S., & Schulman, K. A. (2017). Barriers to Achieving Economies of Scale in Analysis of EHR Data. Applied Clinical Informatics, 08(03), 826–831. https://doi.org/10.4338/ACI-2017-03-CR-0046
Shah, N. H., Milstein, A., & Bagley, P., Steven C. (2019). Making Machine Learning Models Clinically Useful. JAMA, 322(14), 1351–1352. https://doi.org/10.1001/jama.2019.10306
Stamper, J. C., & Koedinger, K. R. (2011). Human-Machine Student Model Discovery and Improvement Using DataShop. In G. Biswas, S. Bull, J. Kay, & A. Mitrovic (Eds.), Artificial Intelligence in Education (pp. 353–360). Springer. https://doi.org/10.1007/978-3-642-21869-9_46
Star, S. L. (2010). This is Not a Boundary Object: Reflections on the Origin of a Concept. Science, Technology, & Human Values, 35(5), 601–617. https://doi.org/10.1177/0162243910377624
Star, S. L., & Griesemer, J. R. (1989). Institutional Ecology, `Translations’ and Boundary Objects: Amateurs and Professionals in Berkeley’s Museum of Vertebrate Zoology, 1907-39. Social Studies of Science, 19(3), 387–420. https://doi.org/10.1177/030631289019003001
Suresh, H., & Guttag, J. V. (2021). A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle. Equity and Access in Algorithms, Mechanisms, and Optimization, 1–9. https://doi.org/10.1145/3465416.3483305
Walonoski, J. A., & Heffernan, N. T. (2006). Prevention of Off-Task Gaming Behavior in Intelligent Tutoring Systems. In M. Ikeda, K. D. Ashley, & T.-W. Chan (Eds.), Intelligent Tutoring Systems (pp. 722–724). Springer. https://doi.org/10.1007/11774303_80
Winne, P. H. (2020). Construct and Consequential Validity for Learning Analytics Based on Trace Data. Computers in Human Behavior, 112, 106457. https://doi.org/10.1016/j.chb.2020.106457
Woo, S. E., Tay, L., Jebb, A. T., Ford, M. T., & Kern, M. L. (2020). Big Data for Enhancing Measurement Quality. In S. E. Woo, L. Tay, & R. W. Proctor (Eds.), Big Data in Psychological Research (pp. 59–85). American Psychological Association. https://doi.org/10.1037/0000193-004
Yarkoni, T., & Westfall, J. (2017). Choosing Prediction Over Explanation in Psychology: Lessons From Machine Learning. Perspectives on Psychological Science, 12(6), 1100–1122. https://doi.org/10.1177/1745691617693393
Zheng, G., Fancsali, S. E., Ritter, S., & Berman, S. (2019). Using Instruction-Embedded Formative Assessment to Predict State Summative Test Scores and Achievement Levels in Mathematics. Journal of Learning Analytics, 6(2), Article 2. https://doi.org/10.18608/jla.2019.62.11
