Would You Trust the Robot?

weirdly accurate.

Would You Trust the Robot?

Would you trust the robot?

This AI trust test uses the judge–advisor method from decision research to find out how far you lean on an AI assistant’s number, and whether leaning makes you more accurate. Researchers have used the same design many times, among them Ilan Yaniv in 2004, with 30 people answering questions about historical dates, and Logg, Minson and Moore in 2019, with 202 people guessing weights from photos. Your result sets your number beside theirs after the last board.

This test runs the same design at a kitchen counter. On each of nine boards a tea towel lifts for a few seconds, 3 at first and 1.7 by the end, over blueberries, sugar cubes, coffee beans and other small things that get more numerous and smaller. You set your count, an AI assistant gives its own, and you can move yours or keep it before the real count is revealed. Your weight of advice is calculated as in Yaniv (2004): how far you moved, as a share of the distance between your first count and the assistant’s. The result shows that number next to the studies and tells you whether leaning on the assistant made your answers better or worse.

It is not a personality test. Nine boards is a small sample, and the studies used other tasks (weights from photos, historical dates, forecasts), so the comparison with them is rough and there is no percentile. No AI model is called while you play: the assistant runs on the page, and nothing you enter is uploaded.

What the test measures

One thing: how far you move toward an AI assistant’s answer, and what that does for your accuracy.

The task

  • Nine boards, from blueberries to peppercorns: each holds more, smaller things, and the look shrinks from 3 seconds to 1.7
  • The judge–advisor design from decision research: your first estimate, the assistant’s count, your final estimate
  • Set your count with a slider; after the assistant speaks, its number and your first count stay marked on the same slider
  • The real count after every board; a final within 15% of it counts as close

Your result

  • Your weight of advice: how far you moved toward the assistant, from 0% (kept your own count) to 100% (took its number)
  • A name for the way you used the assistant, and all nine boards on one scale: your first count, the assistant’s, your final and the truth
  • Who ended up closer to the truth: your first counts, your finals, the assistant, or simply splitting the difference every time
  • The studies’ reference points, shown after the last board: how far people moved toward advice in judge–advisor studies, and toward the same advice labelled as an algorithm’s
  • A shareable card and a challenge link that gives a friend the same nine boards

Before you start

  • About two minutes; free, no sign-up
  • Works on a phone or a computer with 3D graphics (WebGL): drag the knob, tap − and +, or use the arrow keys; sound is optional
  • Nothing you enter is uploaded; a challenge link carries your numbers inside the link, after the # sign

AI trust FAQ

Is this test free? Do I need to sign up to see my result?

Yes, it’s free, and there’s nothing to sign up for: no account, no email, no payment. Your result appears on this page the moment you finish, and you can share it or play again straight away.

Do people trust AI advice more than human advice?

The result sheet answers that after your nine boards, using Logg, Minson and Moore’s 2019 experiments. They gave people the same advice labelled either as an algorithm’s or as other people’s, in tasks such as guessing weights from photos (202 people), ranking songs and predicting romantic attraction, and measured how far each label moved them. The figures wait until the end because knowing them first would change how you weigh the assistant.

How does the AI trust test score me?

The AI trust test uses weight of advice, the standard judge–advisor measure: how far your final count moved from your first count toward the assistant’s, as a share of the gap between them. Keeping your own count scores 0%, taking the assistant’s scores 100%. Boards where you already agreed with the assistant are left out.

What is a normal weight of advice?

The typical figures appear on the result sheet beside your own weight of advice, once the nine boards are done. They come from Ilan Yaniv’s 2004 study, in which 30 people answered 15 questions about historical dates and then saw another person’s estimate, and from Logg and colleagues’ 2019 summary of the research. The page keeps them back because a number to aim for would change what you do.

Is the AI assistant in this test a real AI?

No. No AI model is called while you play: the assistant in the AI trust test is a script that runs on the page, and nothing you enter is uploaded. The result sheet explains exactly how its counts were set once you have finished all nine boards, because knowing that beforehand would change how you treat it.

What research is the AI trust test based on?

The AI trust test uses the judge–advisor design from decision research, scored with the weight of advice measure from Ilan Yaniv (2004). The result sheet also draws on Logg, Minson and Moore (2019), Dietvorst, Simmons and Massey (2015, 361 people), Parasuraman and Manzey (2010) and Soll and Larrick (2009), and says what each found after your last board. Full references are under “Where the numbers come from”.

Should I average my answer with the AI’s?

The AI trust test leaves that choice to you, because how you combine the two numbers is exactly what it measures. Afterwards the result sheet shows how splitting the difference on every board would have done against your first counts, your finals and the assistant, and what Soll and Larrick’s 2009 research on revising judgments says about averaging.

Is this a personality test?

No. The AI trust test records how you used one assistant on nine counting boards. The studies it is compared with used other tasks, such as weights from photos, historical dates and forecasts, so the comparison is a rough reference rather than a ranking.

Where the numbers come from

There is no population data for weight of advice on a counting task like this one, so the result gives no percentile; it sets your number next to the studies’ typical values (Yaniv 2004, Study 1; Logg et al. 2019, Experiment 1A). The site does not publish its own players’ data.

How the score works

Weight of advice

(final − first) ÷ (advice − first), the standard measure from Yaniv (2004), kept between 0 and 1. Staying put is 0, adopting the advice is 1, meeting it halfway is 0.5.

Close

A final count within 15% of the real count. Accuracy is compared as the average error of your first counts, your finals, the assistant and the halfway point.