Can automated feedback improve teachers’ uptake of student ideas? Evidence from a randomized controlled trial in a large-scale online course

Journal article

Dorottya Demszky, Jing Liu, Heather C. Hill, Dan Jurafsky, Chris Piech
Educational Evaluation and Policy Analysis, app, 2023 May

DOI: 10.3102/01623737231169270

View PDF Link <<

Cite

APA Click to copy
Demszky, D., Liu, J., Hill, H. C., Jurafsky, D., & Piech, C. (2023). Can automated feedback improve teachers’ uptake of student ideas? Evidence from a randomized controlled trial in a large-scale online course. Educational Evaluation and Policy Analysis. https://doi.org/10.3102/01623737231169270

Chicago/Turabian Click to copy
Demszky, Dorottya, Jing Liu, Heather C. Hill, Dan Jurafsky, and Chris Piech. “Can Automated Feedback Improve Teachers’ Uptake of Student Ideas? Evidence from a Randomized Controlled Trial in a Large-Scale Online Course.” Educational Evaluation and Policy Analysis (May 2023).

MLA Click to copy
Demszky, Dorottya, et al. “Can Automated Feedback Improve Teachers’ Uptake of Student Ideas? Evidence from a Randomized Controlled Trial in a Large-Scale Online Course.” Educational Evaluation and Policy Analysis, app, May 2023, doi:10.3102/01623737231169270.

BibTeX Click to copy

@article{demszky2023a,
  title = {Can automated feedback improve teachers’ uptake of student ideas? Evidence from a randomized controlled trial in a large-scale online course},
  year = {2023},
  month = may,
  journal = {Educational Evaluation and Policy Analysis},
  school = {app},
  doi = {10.3102/01623737231169270},
  author = {Demszky, Dorottya and Liu, Jing and Hill, Heather C. and Jurafsky, Dan and Piech, Chris},
  howpublished = {},
  month_numeric = {5}
}

Abstract

Providing consistent, individualized feedback to teachers is essential for improving instruction but can be prohibitively resource-intensive in most educational contexts. We develop M-Powering Teachers, an automated tool based on natural language processing to give teachers feedback on their uptake of student contributions, a high-leverage dialogic teaching practice that makes students feel heard. We conduct a randomized controlled trial in an online computer science course (N = 1,136 instructors), to evaluate the effectiveness of our tool. We find that M-Powering Teachers improves instructors’ uptake of student contributions by 13% and present suggestive evidence that it also improves students’ satisfaction with the course and assignment completion. These results demonstrate the promise of M-Powering Teachers to complement existing efforts in teachers’ professional development.