Comparing teaching texts from six large language models: strength in higher-order questions does not mean classroom effectiveness
Evaluating the effects of large language model applications in classroom teaching

English translation
The study compared multidisciplinary teaching texts generated by six large language models with teachers’ instructional materials, using three dimensions: value guidance, knowledge construction and thinking development. The objects of evaluation were texts, not student learning outcomes; the findings cannot establish that large language models teach better than teachers in classrooms.
Provides a concrete research design and clear boundaries around what is evaluated in discussions of large language models for education.
Sources and editorial checks
Editorial check:
- Source type
- Peer-reviewed paper
- Scope
- Research/policy subjects in China
- Study design
- Comparison of teaching texts and content evaluation; not a trial of student outcomes.
- Population and context
- Multidisciplinary teaching texts for primary and secondary education generated by six mainstream models, and teachers’ instructional materials.
- Sample
- Six models; the number of texts has not been verified.
- Intervention and comparison
- Compares model-generated and teacher-created texts on value guidance, knowledge construction and thinking development.
- Main findings
- The study compared multidisciplinary teaching texts generated by six large language models with teachers’ instructional materials, using three dimensions: value guidance, knowledge construction and thinking development. The objects of evaluation were texts, not student learning outcomes; the findings cannot establish that large language models teach better than teachers in classrooms.
- Outcomes and uncertainty
- The abstract proposes three evaluation dimensions; classroom learning outcomes cannot be inferred.
Not yet verified: funding and conflicts of interest. Consult the original source.
Limitations
- The abstract, figures and paper metadata can be checked. The full methods, inter-coder agreement and sampling have not yet been reviewed paragraph by paragraph.
Source: Journal of East China Normal University · xbjk.ecnu.edu.cn