AI Tutors Are Here. The Evidence on Learning Is Not.

Benjamin Bloom's 1984 "2 Sigma Problem" is the canonical argument for personalised tutoring: students who receive one-on-one human tutoring perform two standard deviations better than classroom instruction. If AI can approximate individualised instruction at scale, the implication for human capital development is profound. The question is whether current AI tutoring systems actually deliver meaningfully improved learning outcomes — and the honest answer is that we don't yet know, because the rigorous studies haven't been done.

What AI Tutoring Systems Can Do

Large language models are genuinely good at several things that are pedagogically valuable: explaining concepts in multiple ways, answering follow-up questions without frustration, generating varied practice problems, and providing immediate feedback on answers. They are demonstrably bad at a few things that matter: reliably detecting when a student has a fundamental misconception (as opposed to a surface-level error), maintaining consistent scaffolding strategies across long interactions, and calibrating difficulty without explicit performance data.

Intelligent tutoring systems (ITS) — the older, non-LLM AI tutoring tradition — have a more substantial evidence base. Carnegie Learning's MATHia, developed with decades of cognitive science research, has multiple randomised controlled trials showing positive effects on algebra learning outcomes. The evidence specifically for LLM-based tutoring is much thinner. Duolingo's own published research shows engagement improvements; rigorous learning outcome data is limited. Khanmigo's effectiveness is based primarily on user testimonials and preliminary internal analysis rather than RCTs.

The Implementation Challenge

Even if AI tutoring works in optimal conditions, deployment at scale in under-resourced schools faces different challenges. Students who would most benefit from personalised tutoring — those with weaker academic foundations, less structured home environments, lower digital literacy — are also the students for whom unsupervised AI tutoring is least likely to function effectively. Effective use of AI tutoring tools requires student self-regulation skills that are precisely what struggling students often lack.

Teacher integration is the variable most likely to determine outcomes. Schools that have seen positive results from AI tutoring tools report using them as supplements to teacher instruction, with teachers reviewing AI interaction logs and following up on identified misconceptions. Schools that deploy AI tutors as substitutes for teacher attention — often under financial pressure — are attempting something for which there is no evidence base.

What the Research Would Need to Show

For AI tutoring to be adopted with confidence, the field needs: randomised controlled trials with pre-registered hypotheses and outcome measures, disaggregated results by student characteristics (baseline ability, socioeconomic status, English language proficiency), longitudinal follow-up measuring retention rather than immediate performance, and comparison against low-cost human alternatives (peer tutoring, structured practice) rather than just "no intervention." None of the major AI tutoring platforms have published this level of evidence. The enthusiasm has outrun the science, which is a pattern that education technology has repeated in every previous decade. It may be different this time. But claiming that requires evidence, not theory.