Measuring conversational uptake: a case study on student-teacher interactions


Journal article


Dorottya Demszky, Jing Liu, Zid Mancenido, Julie Cohen, Heather Hill, Dan Jurafsky, Tatsunori Hashimoto
arXiv preprint arXiv:2106.03873, arXiv, 2021 Jun


Link <<
Cite

Cite

APA   Click to copy
Demszky, D., Liu, J., Mancenido, Z., Cohen, J., Hill, H., Jurafsky, D., & Hashimoto, T. (2021). Measuring conversational uptake: a case study on student-teacher interactions. ArXiv Preprint ArXiv:2106.03873. https://doi.org/10.48550/arXiv.2106.03873


Chicago/Turabian   Click to copy
Demszky, Dorottya, Jing Liu, Zid Mancenido, Julie Cohen, Heather Hill, Dan Jurafsky, and Tatsunori Hashimoto. “Measuring Conversational Uptake: a Case Study on Student-Teacher Interactions.” arXiv preprint arXiv:2106.03873 (June 2021).


MLA   Click to copy
Demszky, Dorottya, et al. “Measuring Conversational Uptake: a Case Study on Student-Teacher Interactions.” ArXiv Preprint ArXiv:2106.03873, arXiv, June 2021, doi:10.48550/arXiv.2106.03873.


BibTeX   Click to copy

@article{demszky2021a,
  title = {Measuring conversational uptake: a case study on student-teacher interactions},
  year = {2021},
  month = jun,
  journal = {arXiv preprint arXiv:2106.03873},
  publisher = {arXiv},
  doi = {10.48550/arXiv.2106.03873},
  author = {Demszky, Dorottya and Liu, Jing and Mancenido, Zid and Cohen, Julie and Hill, Heather and Jurafsky, Dan and Hashimoto, Tatsunori},
  howpublished = {measurement},
  month_numeric = {6}
}

Abstract

In conversation, uptake happens when a speaker builds on the contribution of their interlocutor by, for example, acknowledging, repeating or reformulating what they have said. In education, teachers' uptake of student contributions has been linked to higher student achievement. Yet measuring and improving teachers' uptake at scale is challenging, as existing methods require expensive annotation by experts. We propose a framework for computationally measuring uptake, by (1) releasing a dataset of student-teacher exchanges extracted from US math classroom transcripts annotated for uptake by experts; (2) formalizing uptake as pointwise Jensen-Shannon Divergence (pJSD), estimated via next utterance classification; (3) conducting a linguistically-motivated comparison of different unsupervised measures and (4) correlating these measures with educational outcomes. We find that although repetition captures a significant part of uptake, pJSD outperforms repetition-based baselines, as it is capable of identifying a wider range of uptake phenomena like question answering and reformulation. We apply our uptake measure to three different educational datasets with outcome indicators. Unlike baseline measures, pJSD correlates significantly with instruction quality in all three, providing evidence for its generalizability and for its potential to serve as an automated professional development tool for teachers.

Share

Tools
Translate to