How Overconfident Are You?

weirdly accurate.

How Overconfident Are You?

    Am I overconfident?

    This overconfidence test uses the classic 90% confidence interval task to find out whether your “90% sure” really comes true nine times in ten. For ten numeric questions you stretch a net until you are 90% sure the answer is inside, and only when all ten are cast do the answers drop in. Researchers have run the same task for decades, including McKenzie, Liersch and Yaniv, who gave 135 people 40 questions each in 2008; the result sheet shows how your catch compares.

    This overconfidence test uses the same interval task. You get ten numeric questions — dates, sizes and distances, counts — drawn at random from a bank of 55 long-settled facts, each checked against a listed source. For each one you drag a net along a line until you’re 90% sure the answer is inside it. Nothing is revealed until all ten are cast, so one miss can’t teach you to widen the next; then the answers drop in. Ten true 90% ranges should catch about nine, and the gap between nine and your catch is your overconfidence. Because a net across the whole line would catch everything, each net also scores up to 10 points with the interval score used for forecast ranges: tighter nets score more, a fish outside costs points, and a net covering three quarters of the line or more scores nothing. The result also shows how likely a catch as small as yours would be for someone who really is 90% sure, and how much wider your nets needed to be.

    The hit rate measures calibration, not knowledge or intelligence: you don’t need to know any answer, only how unsure you are. The points also reward knowing enough to keep nets tight. Ten nets is a small sample — even a perfectly calibrated person catches 8 or fewer about a quarter of the time. The questions are not the ones used in the studies, so the comparison with them shows the pattern, not a ranking.

    What the test measures

    One thing: whether your “90% sure” comes true nine times in ten.

    The task

    • The interval task from overprecision research: for each of ten numeric questions, a range you are 90% sure holds the true value
    • Ten questions a run — four dates, three sizes, distances or temperatures, three counts — drawn at random from a bank of 55 and shuffled
    • Press on the line and drag to lay a net; drag either buoy or slide the whole net to adjust. While you lay it, the net shows what it’s worth, and greys out once it covers the whole sea. The buoys also work from the keyboard
    • No feedback until all ten nets are cast; then each answer drops in and the nets that caught it turn gold

    Your result

    • Points out of 100: each net scores up to 10, less the wider it is; a fish outside costs points, and a net over three quarters of the line or more scores 0 and isn’t a catch
    • How many of your ten nets caught the answer, set against what a true 90% would catch
    • One of five bands: 9–10 “Your 90% means 90%”, 7–8 a touch overconfident, 4–6 overconfident, 3 or fewer very sure, often wrong — and “playing it too safe” when nets cover the whole sea or are far wider than the catch needed
    • The chance that someone really 90% sure would catch as few as you did, and how many times wider each net needed to be for nine to catch
    • For comparison, what the studies found when people gave 90% ranges (McKenzie et al. 2008; Moore et al. 2015)
    • A catch log with every answer and its source, a share card that shows your total but none of the answers, and a link that challenges a friend

    Before you start

    • About three minutes
    • Phone or computer; on a computer the buoys also move with the arrow keys
    • Sound is optional and can be switched off
    • Free, no sign-up. A challenge link carries your name and score inside the link itself, after the #, and isn’t uploaded

    Overconfidence FAQ

    Is this test free? Do I need to sign up to see my result?

    Yes, it’s free, and there’s nothing to sign up for: no account, no email, no payment. Your result appears on this page the moment you finish, and you can share it or play again straight away.

    How do you test if you’re overconfident?

    Ask for ranges with a stated confidence and count how often they hold the answer. That is the interval task from overprecision research, and it is what this overconfidence test uses: ten numeric questions, a 90%-sure net for each. If your 90% ranges catch far fewer than nine of ten answers, your confidence is running ahead of your accuracy.

    How is the overconfidence test scored?

    The overconfidence test gives two numbers. The hit rate counts how many of your ten nets caught the true answer and compares that with the nine a truly 90%-sure person would catch on average, with the binomial chance of a catch as low as yours. The points, out of 100, add how wide the nets were: each net scores up to 10, less the wider it is, a fish outside costs 20 times its distance, and a net over three quarters of the line or more scores 0. Answers stay hidden until all ten nets are out.

    What is a normal score on a 90% confidence quiz?

    The result sheet shows that after all ten nets are in, so your own nets aren’t steered by it. It quotes the hit rates reported in Moore, Tenney and Haran’s 2015 review of interval-estimate studies, and in McKenzie, Liersch and Yaniv’s 2008 study of 135 people answering 40 questions each. Those studies used other questions, so read the comparison as a pattern, not a ranking.

    Where do the overconfidence test’s questions come from?

    The overconfidence test draws ten questions at random from a bank of 55 long-settled facts: four dates, three sizes, distances or temperatures, and three counts each run. Every answer is checked against a listed source, and the catch log on the result sheet shows each answer with its source. You don’t need to know any of them: the test asks only how sure you are.

    Are experts less overconfident?

    The result sheet answers that after your ten nets, from McKenzie, Liersch and Yaniv’s 2008 study, in which 135 people, experts and novices among them, gave 90% ranges for 40 questions each. Knowing more about a subject lets you keep a net tight; whether it also keeps a 90% range honest is what the study tested.

    Does a low score mean I don’t know much?

    Not the hit rate. That part measures calibration — whether your sense of “sure” matches what happens — not how much you know: honestly wide nets keep your hit rate near nine in ten without knowing a single answer. The points do reward knowledge as well, because knowing more lets you keep nets tight. Ten nets is also a small sample: even someone perfectly calibrated catches 8 or fewer about a quarter of the time.

    Can’t I just make every net huge?

    It doesn’t pay. A net that covers three quarters of the line or more can’t miss, so it greys out while you lay it, scores 0 and doesn’t count as a catch. Wide nets just under that still score little, because every net’s points shrink with its width. The best bet is the net you honestly are 90% sure of — just wide enough that you would bet nine to one on it.

    Where the numbers come from

    Your catch is compared with a true 90% using the binomial distribution, and with the hit rates these studies report; the points use the interval score from Winkler and from Gneiting and Raftery. The questions here differ from theirs, so there is no percentile. Every answer in the question bank lists its own source in the catch log. The site does not publish its own players’ data.

    How it’s scored

    Hit rate

    The share of your ten 90% ranges that contain the true answer. Perfect calibration is about nine in ten.

    Points (interval score)

    Each net scores 10 × (1 − width − 20 × distance outside), measured as shares of its line and floored at 0. This is the interval score for 90% ranges (Winkler 1972; Gneiting and Raftery 2007): width costs, and a miss costs 2 ÷ 10% = 20 times its distance, so stretching a net to be safe doesn’t pay.

    Whole-sea net

    A net covering three quarters of its line or more. Every answer sits between a tenth and nine tenths of its line, so such a net can’t miss: it scores 0 and doesn’t count as a catch.

    Overprecision

    Ranges narrower than your stated confidence allows: the form of overconfidence this task measures, reviewed by Moore, Tenney and Haran (2015).

    Calibration check

    For someone truly 90% sure, the number caught out of ten follows a binomial distribution: 9 or 10 about three times in four, 8 or fewer about a quarter of the time. Your result says how often a catch as low as yours would happen by luck alone.

    Widening factor

    How many times wider each of your nets would have had to be for nine of the ten to catch.