Skip to content
Interface language
Journal of East China Normal University·(Date only in original)Featured

Comparing teaching texts from six large language models: strength in higher-order questions does not mean classroom effectiveness

Evaluating the effects of large language model applications in classroom teaching

Topic illustration of education researchers examining charts and notes
Topic illustration · not an event photograph
Reading guide

English translation

The study compared multidisciplinary teaching texts generated by six large language models with teachers’ instructional materials, using three dimensions: value guidance, knowledge construction and thinking development. The objects of evaluation were texts, not student learning outcomes; the findings cannot establish that large language models teach better than teachers in classrooms.

Why read this

Provides a concrete research design and clear boundaries around what is evaluated in discussions of large language models for education.

Sources and editorial checks

Editorial check:

Source type
Peer-reviewed paper
Scope
Research/policy subjects in China
Study design
Comparison of teaching texts and content evaluation; not a trial of student outcomes.
Population and context
Multidisciplinary teaching texts for primary and secondary education generated by six mainstream models, and teachers’ instructional materials.
Sample
Six models; the number of texts has not been verified.
Intervention and comparison
Compares model-generated and teacher-created texts on value guidance, knowledge construction and thinking development.
Main findings
The study compared multidisciplinary teaching texts generated by six large language models with teachers’ instructional materials, using three dimensions: value guidance, knowledge construction and thinking development. The objects of evaluation were texts, not student learning outcomes; the findings cannot establish that large language models teach better than teachers in classrooms.
Outcomes and uncertainty
The abstract proposes three evaluation dimensions; classroom learning outcomes cannot be inferred.

Not yet verified: funding and conflicts of interest. Consult the original source.

Limitations

  • The abstract, figures and paper metadata can be checked. The full methods, inter-coder agreement and sampling have not yet been reviewed paragraph by paragraph.

Source: Journal of East China Normal University · xbjk.ecnu.edu.cn