Getting a question wrong might be the best thing that can happen to a student, if they try again.
Researchers at Carnegie Mellon University’s School of Computer Science (SCS) revealed how small design changes in online tutoring platforms can help students push through their mistakes and keep learning.
The research team made small tweaks to things like text and color to see how these alterations improved students’ persistence.
“Being wrong tends to be a very strong signal that people interpret as, ‘This is not working, I’m going to stop,'” said Paulo Carvalho(opens in new window), an assistant professor in the Human-Computer Interaction Institute(opens in new window) (HCII). “In fact, it’s the opposite. If you’re getting a practice question wrong, you’re probably going to get the feedback you need to understand your mistake in the next step. But learners usually interpret being wrong as, ‘I’m not learning,’ and we stop. But if you stop, then you definitely don’t learn. This research looks at that issue.”
The CMU team partnered with the South African nonprofit Siyavula(opens in new window), which offers an intelligent AI-based tutoring system, to test these interventions. The partnership was made possible through HCII Associate Professor of Learning Sciences Amy Ogan’s(opens in new window) Learning Sciences for Innovators(opens in new window) program.
The researchers’ paper, “Will They Try Again? A Large-Scale RCT on Scaffolds that Support Persistence in an Intelligent Tutoring System,(opens in new window)” received an honorable mention at the recent Association for Computing Machinery Conference on Human Factors in Computing Systems (CHI 2026).
Michael Asher(opens in new window), a project scientist in the HCII, said the team looked to psychology-inspired design to create the two changes: a written prompt and a nudge. The written prompt appeared after the student got a question wrong, encouraging them to try again. The nudge, which also appeared after an incorrect answer, provided a visual cue prompting a user to choose a certain option — something familiar to nearly everyone because it’s baked into human-computer interaction design (for example, when users close a Microsoft Word document, the “Save” button is highlighted in blue). In this research, the button that read “Try an exercise like this again” was highlighted in bright orange, nudging students to try again.
“We move through the world making decisions and we process tons of information, so we rely on a lot of shortcuts,” Asher said. “Default options are powerful ways to simplify things, because we assume that they’re defaults for a reason. So there’s a whole industry of nudge-based interventions, which often change defaults to help someone make a decision that is desired, hopefully for beneficial reasons.”
This study was a randomized controlled trial involving about 160,000 students who used Siyavula’s online tutoring system to complete 17 million practice problems. Researchers randomly assigned some students to receive the nudge, the written prompt, neither or both. They found that the prompt and nudge increased how often students persisted after failure by 2% and 9%, respectively.
These increases represent more than just statistics: They are thousands of instances of people learning after getting a problem wrong. Stacking the prompt and nudge together had a powerful additive effect, leading students to persist 11% more often than those who received no help at all.
“What we found here — and this is something that was only possible because we were able to work with a really large sample to test this precisely — is that these two interventions actually stack on top of each other really nicely,” Asher said. “The prompt seems to make people more likely to try again, and the nudge has an even larger effect. When you put them together, it’s the sum of the parts, which we can really see because of our sample size.”
Along with Asher, Ogan and Carvalho, the research team included HCII doctoral student Yumou Wei and the Siyavula Foundation’s Adam Reynolds. Learn more in the research paper on the CHI 2026 website(opens in new window).
“Carnegie Mellon University is a private research university in Pittsburgh, Pennsylvania. The institution was originally established in 1900 by Andrew Carnegie as the Carnegie Technical School. In 1912, it became the Carnegie Institute of Technology and began granting four-year degrees.”
Please visit the firm link to site