At Harvard, physics students who learned a topic from an AI tutor at home scored higher than classmates taught the same material in a well-run active-learning class, and they got there in less time. At the University of Maryland, students in course sections given an AI tutor ended the term with lower final grades than students in other sections of the same course.

So do AI tutors work? We read 21 controlled studies published between 2024 and October 2026, from Harvard to Ghana, and compared how each tool was designed with what happened to learning. The short answer is yes, when the tutor makes students do the thinking and has a fixed place in their work. The design decides which way it goes.

What the studies show, in five points.

• AI that hands out answers can make students learn less. Every study where learning fell gave students a tool that answers on request.
• The best tutors are built around a lesson. Tutors that walked students through prepared material beat their comparison group every time. Tutors that were only told to give hints mostly avoided harm and added little.
• Better homework is not proof of learning. Scores always rose while students had the AI. What matters is how they do once it is gone.
• Students rarely use a tutor they don’t have to. Optional tools did not move grades. Teaching students how to use one did.
• Expect modest gains over a school year. The big numbers come from short studies. Year-long effects on standardised tests were small.

1. AI that hands out answers can make students learn less

Four studies found that students learned less with AI than without it. In all four, the tool gave answers on request.

• Turkey, nearly 1,000 high-school maths students. In a PNAS trial, students practised with an open ChatGPT-style chat. On the exam afterwards, taken without AI, they scored 17% lower than students who practised with their notes and textbook.
• Brazil, 120 business students. Students who studied with unrestricted ChatGPT remembered less on a surprise test 45 days later (57.5% correct against 68.5%).
• China, 26,811 secondary students. Once students started using chatbots such as Doubao and DeepSeek for homework, their closed-book exam scores fell by about a fifth within six months.
• Maryland, 2,379 university students. The course tutor drew on instructors’ own materials, but most instructors left it in its default mode, which gives answers and explanations directly. In sections of the same course, final grades came out about 4 points lower out of 100, and students took far less part in course activities.

Maryland needs one caveat. Only about 15% of students ever used the tutor, and across all courses the grade drop was not clear, so part of the effect may come from how students changed their habits once it arrived. First-generation students lost the most.

How students use an open chatbot matters too. In a Cologne study of Python learners and a Middlebury College study, students who asked the AI to explain did better than students who asked it for solutions or had it write for them.

This pattern is consistent across very different settings, though the Chinese study was not a randomised trial and the Brazilian one was small.

2. The best tutors are built around a lesson

Telling a model to give hints instead of answers removes most of the harm. On its own, it rarely adds much.

Seven tutors in the set were prompted that way, including Khan Academy’s Khanmigo in Tennessee and the “GPT Tutor” in the Turkish trial. None lowered learning, and most made no clear difference. The Turkish tutor brought exam scores back to the level of students with no AI. Over two years, Khanmigo produced a small gain, about what the same practice platform delivers without AI.

Four tutors went further and walked students through prepared material step by step. All four beat their comparison group.

• Harvard physics. The tutor worked from pre-written step-by-step solutions and took students through each part of a problem in order. The typical student’s learning gain was more than double the gain in the active-learning class.
• Medly, GCSE science revision in England. Structured lessons, with answers marked against exam-board criteria. Students outscored peers who revised on their own, in biology, chemistry and physics.
• Rori, maths in Ghana. More than 500 short lessons, each giving a hint first and then a worked solution. Pupils’ maths scores grew by 5.1 points out of 35, against 2.1 for pupils without it.
• An AI lecture agent at Tsinghua. It delivered the course modules and answered questions as they came up, and students scored higher than with a live human lecture or a video course with chatbot support.

What these four share is prepared content that the AI works inside. Feeding a chatbot the course materials is not enough on its own. Two semester-long trials of course-grounded chatbots, at Maryland and in a US marketing course, found a loss and no change.

All four structured tutors succeeded, but their studies were short and used tests written for the study, and two were funded or co-written by the company behind the tool.

3. Better homework is not proof of learning

Every study that measured students both with the AI and without it found the same split. Performance went up while the AI was there. Learning, measured afterwards, often did not.

• Turkey. Practice scores rose 48% with the open chat and 127% with the hint-giving tutor. Exam scores without AI then fell for the open-chat group and stayed flat for the tutor group.
• China. Homework scores rose 18% and homework took 30% less time, while exam scores fell.
• Carnegie Mellon, discrete maths. Homework scores rose with an AI proof tutor. Exam scores did not.
• A writing lab study. Students with AI help produced better revised essays but learned no more about the topic.

For anyone judging an AI tutor, this is the most practical finding in the set. Usage dashboards and in-tool scores measure how students do with the AI switched on, and those numbers almost always look good. Only a test taken without the tool shows learning.

This is the strongest finding here, repeated across four studies with different ages, subjects and designs.

4. Students rarely use a tutor they don’t have to

Buying access is easy. Getting students to use the tool well is the hard part, and the studies show how wide that gap can be.

• Maryland. About 15% of students ever opened the course tutor, for about four sessions each.
• A US marketing course. The tutor never drew regular use, and grades did not change.
• Chile. An email campaign encouraging students to use their course AI raised use but left midterm scores unchanged.
• Khanmigo. Even inside required practice time, the typical student messaged the tutor on a third of practice days, and in only 17% of the sessions where they got something wrong.

One study shows what does work. In a second Chilean trial, an econometrics instructor showed students how to use the course AI as a tutor, for example by asking it to guide them step by step and to hold back the final answer. Final-exam scores rose and the pass rate went up by 12 percentage points.

Four studies agree that optional access does little, and one well-run trial shows that teaching students how to use the tool helps.

5. A human in the loop is the safest bet so far

The most encouraging results for schools that already run tutoring come from AI that works through a person.

• Tutor CoPilot, a US school district. The AI suggests teaching moves to human tutors and never talks to students. Students became 4 percentage points more likely to master the session’s topic, and 9 points more likely with less experienced tutors, at about $20 per tutor per year.
• Eedi, five UK secondary schools. A teacher approved or edited every message the AI drafted. The supervised AI did as well as human tutors and better than pre-written hints.
• Nigeria, after-school English. Students worked with Microsoft Copilot while teachers supervised every session and set the opening prompts. Students scored higher in English, including on their school’s own end-of-term exam.

This one rests on only three studies, none of them peer reviewed yet, and the Nigerian sessions were also extra lesson time.

6. Expect modest gains over a school year

The headline numbers in AI tutoring research mostly come from short studies with tests the researchers wrote themselves. Harvard measured one lesson. Medly ran for four weeks.

Over a full school year, on a standardised test, the effects were small. Across 18 Tennessee middle schools and two years, Khanmigo raised maths scores by about 1.3 national percentile points per term. The Tutor CoPilot authors estimate a barely measurable effect on the end-of-year test.

That is the normal pattern for school interventions of any kind. Plan a pilot for a modest effect, run it for at least a term, and measure it on a test the AI cannot help with.

This rests on the two largest long-running trials and the pattern across all 21 studies.

What this means if you are buying or building an AI tutor

The findings turn into five questions for any vendor, and the same five work as design rules for a product team. Send them in writing before a demo, and ask for the evidence behind each answer.

Five questions to ask before you buy

• Can students get the answer on request, and who decides? Answer-giving tools produced every loss in these studies.
• Is the tutoring built around prepared content, such as worked solutions or a lesson sequence, or only around a prompt? Structure produced the consistent gains.
• What evidence do you have on a test taken without the tool? In-tool scores rise almost regardless.
• Where does it sit in required work, and how will students be taught to use it? Optional access did not move grades.
• Can teachers see and steer what students ask? AI working through a teacher or tutor had the best record.

Which deployment model fits

How we read this as builders

We build and run AI learning products, including AI Tutor and Socratic Owl, with around 90,000 monthly users across our products.

Over the last three months, a small team of ours has built a full AI learning platform for a European edtech company. It has separate student, teacher, parent and school admin roles, an AI tutor and homework assistant grounded in the national curriculum, lessons, and tests with automated grading, all running on the client’s own infrastructure.

Read as builders, these studies point to three decisions to settle before anything else. What the AI may hand over, what prepared content it works inside, and how a pilot will measure learning without it. Our methodology and past projects show how we approach that work, and our AI consulting services cover both the build and the pilot design.

What we don’t know yet

• Long-term effects. Only one trial ran for two years.
• Young children. Only one study included primary-age pupils.
• Subjects beyond maths and science. Most studies taught STEM.
• Cost-effectiveness. Only three studies report costs.
• Independence. Several positive studies involved the company behind the tool.

Two large trials are under way, a full-scale Rori trial in Ghana and a bigger Eedi trial in UK schools. We will add them to the table when they report.

How we put this together

We searched research repositories and journals on 20 October 2026 for randomised and quasi-experimental studies of generative-AI tutors since 2023. We kept studies with school or university students, at least 50 learners, a learning test taken without the AI, and a full text anyone can read. That left 21 studies. We read each paper, recorded how the tool was designed and used, and compared results by design instead of averaging them. Working papers and preprints, which have not yet been peer reviewed, are marked in the table. The table below links each paper where we could. The downloadable version (CSV) links all 21 and adds the exact figures and where each one appears in the paper.

The 21 studies

The Bottom Line

AI tutors work when they make students do the thinking inside required work. Tools that hand out answers produced every loss in these studies, and tutors built around a lesson or working through a human produced the clearest gains. Judge any tool on a test taken without it, over a term or longer, and expect a modest effect.

Take the five vendor questions above into your next procurement conversation, and use the study table, or the CSV download above it, to find the trials closest to your own schools.

Planning an AI tutor for your schools? Tell us what you want to build, or write to [email protected].

FAQ

Are AI tutors as good as human tutors?

In one UK trial, an AI tutor whose every message a teacher approved did as well as human tutors and better than pre-written hints. Tutor CoPilot shows AI making human tutors better, especially less experienced ones. None of the 21 studies compares a fully independent AI tutor with one-to-one human tutoring over a full term, so that question is still open.

Can AI tutoring harm learning?

Yes, when the tool hands out answers. Four studies found students learned less, in Turkey, Brazil, China and at the University of Maryland, and every one involved a tool that gave answers on request. Students did better while using the AI and worse once it was gone.

What makes an AI tutor effective?

Four things show up in the studies that worked. The tutor is built around prepared content such as worked solutions or a lesson sequence, it sits inside required work, students are taught how to use it, and results are measured on a test taken without the AI. A prompt telling the AI to give hints mostly prevents harm without adding much.

Is there evidence for younger students?

Thin. Rori in Ghana covered grades 3 to 9 and found clearly higher maths growth. Khanmigo and the NUMI platform tested middle-school maths in Tennessee, with a small gain and no clear change. No study in this set covers the first years of primary school.

How should a school pilot an AI tutor?

Run it for a term or longer, with a comparison group that does not get the tool. Measure learning on a test students take without the AI, such as the school’s own exams. Track how often students actually use it, because optional tools often go unused. Plan for a modest effect, since year-long gains on standardised tests were small.

Do AI tutors work for university students?

The results are mixed. The Harvard physics tutor, a Berlin lab study, the Tsinghua lecture agent and Chile’s guidance trial all found gains. Course chatbots at Maryland and in a US marketing course found a loss and no change, and unrestricted ChatGPT lowered retention in Brazil. The design and the tool’s place in the course decide which way it goes.