AI Tutoring Reality Check: What the Research Shows

Here is the AI tutoring reality check nobody in L&D wants to sit with: in a June 2026 Stanford study, AI beat 16 law professors in 75 percent of nearly 3,000 blind matchups. Two weeks later, the founder of the world's most famous AI tutor admitted that his first version was, for most students, "a non-event." Both things are true at the same time, and the gap between them is the single most useful thing you can understand before you spend a dollar on an AI tutor for your team.

The lesson is not that AI is overhyped, and it is not that AI is magic. It is that the quality of an AI's answers and the amount your people actually learn are two completely different measurements. Below we walk through the two headlines, explain why they only look like a contradiction, and lay out what the research says separates an AI tutor that moves the needle from one that quietly gets ignored.

Did AI really beat 16 law professors?

Yes. A Stanford Law School study released in June 2026, titled "Law Professors Prefer AI Over Peer Answers," found that AI-written responses won roughly 75 percent of nearly 3,000 anonymized head-to-head matchups against answers written by experienced legal academics.

The setup was deliberately hard to dismiss. Researchers took 40 contract-law questions, the kind a student might raise after class or during office hours, and collected answers from 16 law professors across multiple U.S. law schools alongside answers from AI tools including Google's NotebookLM. Other professors then judged the pairs blind, with no idea which answer came from a human and which from a machine.

The safety numbers were the surprise. Only 3.5 percent of the AI answers were flagged as potentially harmful or misleading, compared with 12 percent of the professors' answers, according to the Stanford Law School write-up.

Lead researcher Julian Nyarko was careful about what this proves. "Our study evaluates the quality of answers given by AI tools," he noted. "But how to implement these tools to most effectively improve student learning is still an open question." His conclusion cuts both ways: the results do not justify wholesale adoption, but "blanket skepticism may be equally unwarranted."

Hold onto that sentence. It is the whole game.

Why did Khan Academy's AI tutor fall flat?

Because a great answer that no one engages with teaches no one anything. That is the blunt takeaway from Sal Khan's own reckoning with Khanmigo, the AI tutor Khan Academy launched in 2023 to enormous fanfare.

"For a lot of students, it was a non-event. They just didn't use it much," Khan told Chalkbeat. He later clarified that he was describing the first version, which "did not change student learning as much as many of us hoped it would."

The problem was not the AI's intelligence. It was the interaction. The original Khanmigo mostly sat and waited for a student to ask it something, and it turns out that waiting is a terrible strategy.

• Students did not initiate. As Khan Academy's chief learning officer Kristen DiCerbo put it, "Students aren't great at asking questions well."
• Passive access changed nothing. Simply having the tutor available did not pull anyone in. Too many learners who could use it never even tried.
• Refusing to just give answers frustrated people who had not engaged enough to know what they were even stuck on.

So Khan Academy rebuilt it. The new Khanmigo is woven directly into practice problems rather than sitting beside them, it prompts learners to explain their reasoning instead of waiting to be asked, and it uses data on recent performance to decide when to jump in, per EdTech Innovation Hub. "The AI could not just sit next to the content," Khan said. "It had to be woven into it."

Why answer quality is not the same as learning

Answer quality measures the AI. Learning measures the human. The Stanford study proves an AI can produce a better contract-law answer than a tenured professor. It says nothing about whether a learner who reads that answer remembers it, understands it, or can apply it next week.

This is where a pile of education research quietly agrees. A Penn State education scholar reviewing the evidence for Phys.org put it plainly: "There is no clear evidence that AI or other kinds of computer-based tutors are superior to human tutors."

The supporting facts are worth knowing:

• A 2020 National Bureau of Economic Research review found that human tutoring produced consistent learning gains across subjects and age groups.
• A 2025 Harvard study found students using an AI physics tutor felt more motivated, but those students were already highly motivated with strong study skills, and the AI was built by the instructors teaching the course.
• Isabelle Hau of the Stanford Accelerator for Learning stresses that students need "relational intelligence" to flourish, because learning is a social endeavor at its core.

None of that means AI tutors are useless. It means the AI is not the active ingredient. The active ingredient is whether the learner does the cognitive work: retrieving, explaining, practicing, and getting feedback. A brilliant answer delivered to a passive brain is a lecture with extra steps, and lectures are exactly what we already know people forget.

What actually makes an AI tutor work

An AI tutor works when it forces engagement instead of offering it. Every success story, including Khanmigo's own turnaround, points at the same design principles. If you are evaluating a tool, this is your checklist.

1. It is embedded where the work happens. The tutor lives inside the task, the module, or the practice problem, not in a separate chat window the learner has to remember to open.
2. It makes people produce, not just consume. It asks the learner to explain their reasoning, answer a question, or attempt a solution before it responds.
3. It adapts to the individual. It uses signals like recent performance and mistakes to decide what to surface next, rather than treating everyone identically.
4. It gives feedback in the moment. Correction lands while the attempt is still fresh, which is when it actually sticks.
5. It drives motivation, not just access. It pulls the learner in with prompts, checks, and relevance instead of waiting to be summoned.

Notice what every item has in common. They are all about interaction, not about how smart the model is. The Stanford result and the Khanmigo flop stop looking contradictory the moment you accept that intelligence sits upstream of learning, and interaction is the thing that carries it across the gap.

What the AI tutoring reality check means for corporate training

For L&D teams, the AI tutoring reality check lands as a purchasing rule: do not buy an AI that answers questions, build an experience that makes people do the work. The exact same trap that snared Khanmigo v1 is waiting in corporate training, where the default failure mode is already passive video that employees click play on and tune out.

Bolting a chatbot onto a training library recreates the "non-event" precisely. Nobody opens it, nobody asks it good questions, and the completion dashboard stays green while retention stays flat.

The alternative is to make interaction the default rather than an option. That is the whole premise behind interactive training videos: instead of a video that talks at a learner for ten minutes, an AI tutor pauses to ask questions, adapts to the answers, and responds in real time, so the employee is producing and explaining, not just watching.

Three questions turn the research into a buying decision:

• Does it make the learner produce, or just consume? If your people can finish without ever answering anything, you have bought a lecture.
• Does it adapt to each person? One path for everyone is the "one-size-fits-all" model that every AI-education advocate rightly complains about.
• Can you measure learning, not just completion? Watch whether performance improves on the next attempt, the same signal Khan Academy now tracks, rather than counting who clicked through.

Get those three right and AI tutoring stops being a gamble. Get them wrong and you have paid for a very expensive machine that most of your team will never talk to.

FAQ

Do AI tutors actually improve learning outcomes?

They can, but only when they are designed around active engagement. Research shows AI can produce excellent answers, yet learning gains depend on whether the tutor makes the learner retrieve, explain, and practice rather than passively read. The technology is necessary but not sufficient.

Are AI tutors better than human teachers?

Not according to current evidence. A Penn State review found no clear proof that AI or computer-based tutors outperform human tutors, and researchers stress that learning is deeply social. The most effective setups use AI to support human coaching and practice, not to replace it.

How do I choose an AI tutor for corporate training?

Judge it on interaction, not intelligence. Ask whether it embeds into the actual work, prompts learners to produce answers, adapts to each person, and lets you measure performance gains instead of just completion rates. If it only waits to be asked questions, expect the same low engagement that stalled early AI tutors.

Passive content is where corporate learning goes to die, and a smarter machine talking at people does not change that. The 2026 research all points the same direction: intelligence is cheap now, and the scarce ingredient is interaction. Build training that makes people think, respond, and practice, and the AI finally becomes what everyone promised it would be.