The argument in brief
The question
What challenges do we give ourselves where we accidentally create a negative score for each failure?
The claim
Setting an outcome benchmark turns every shortfall into lost points, which drains motivation and makes us quit. Measuring the actions we control and the skills we learn keeps us going, and when we have stopped learning it is time to pivot. Confidence: A practical recommendation from my own experience as a creator. One talk reporting an informal experiment is cited; no studies.
The evidence
  • Mark Rober's TEDxPenn talk (2018), which reports an online coding puzzle where players who lost points for failed attempts succeeded less often and made far fewer attempts than those who didn't. It is his own experiment with his audience, presented in a talk, not a peer-reviewed study.
  • My own experience of losing subscribers on some videos in the earlier years of my YouTube channel.
  • My observation of a few people who stopped publishing after their follow-up work didn't match an early success.
The best counter
Outcome targets can be useful information, not just pressure. And sometimes what's needed isn't more learning or changing strategy but patience with the same approach until results arrive, so 'am I still learning?' could lead people to pivot too early or too late.
What would change my mind
Evidence that people who track outcomes against a benchmark persist and improve as much as those who track their actions and skills, or that the Mario Effect doesn't hold up in better-controlled studies.

The questionWhat challenges do we give ourselves where we accidentally create a negative score for each failure?I was watching an interview with Mark Rober on the Colin and Samir YouTube channel and he mentioned the Mario Effect. It’s the concept that if you’re faced with a learning challenge and you lose points for each failure, you’ll be less likely to learn and will quit sooner.SourceRober describes a coding puzzle his YouTube audience took part in, with two randomly served versions: in one, a failed attempt cost five of a starting 200 points; in the other it didn’t. He reports a success rate of around 52% for the penalised players and 68% for the rest, from 50,000 data points, and says those who weren’t penalised made nearly two and a half times more attempts. It is his own informal experiment, reported in a talk; I haven’t found it written up or peer reviewed, and the players were self-selected viewers of his channel.The Super Mario Effect: tricking your brain into learning more, Mark Rober, TEDxPenn, April 2018

So it got me thinking. What challenges do we give ourselves where we accidentally create a negative score for each failure?

The benchmark we set

I’m a content creator. I create YouTube videos, this newsletter and various other things. When thinking of publishing content, sometimes I consider what impact that content will have. Will it get the views, likes, retweets?

An assumptionI’m assuming that five fake internet points in a puzzle work on us the same way as a missed target for views or revenue. The stakes are very different, and the talk doesn’t test that.This hypothesis of setting the success criteria puts a baseline on our score. Anything higher than this is a bonus. And then you look at your video’s performance in the dashboard. You see a video with fewer views than normal, underperforming subscriber growth and revenue.

One video, two ways of keeping score

This video had 1,866 views in the 306 days since it was published.

02.5K5K7.5K0102204306 daystypical1,866
This videoTypical performanceShort of typical: points lost

Score against the benchmark

−1,080

views below the low end of typical, at day 306

What the dashboard said

  • Views1.9K1.0K less than usual
  • Watch time123.7 hoursAbout the same as usual
  • Subscribers+18
  • Estimated revenue£5.01
Views of one of my YouTube videos over its first 306 days (line) against the range YouTube showed as typical (grey band), with the dashboard's four headline figures. All values come from a screenshot of my YouTube Studio dashboard; the line and band are traced by eye, so they are approximate. The score is my framing, not a dashboard figure: views short of the low end of typical, counted as points lost.
Describe this animation

A line chart of one of my YouTube videos over the 306 days after it was published, with values traced by eye from a screenshot of my YouTube Studio dashboard. The video's views rise quickly to about 500 in the first couple of days, then climb slowly to 1,866 by day 306. A grey band shows what YouTube called typical performance; it widens over time and ends between roughly 2,950 and 6,180 views. For about the first 85 days the line sits at or just above the bottom of the band; after that it falls further and further below it.

In the first view, "Score the outcome", the line draws from day 0 to day 306 while a score ticks down. The score counts the views the video is short of the low end of typical as points lost: it stays at 0 for about 85 days, then falls to about −1,080 by day 306. The gap is shaded coral. Beside it are the dashboard's own headline figures: views 1.9K, "1.0K less than usual"; watch time 123.7 hours, "about the same as usual"; subscribers +18; estimated revenue £5.01.

A two-option toggle switches to "Score what I learned". The chart stays the same but the coral gap disappears and points lost are not counted. Instead the panel asks the essay's questions. Strategy: is a 3-minute or a 15-minute video better; just voiceover, or live camera mixed with b-roll; which kinds of hook are worth trying. Skills: am I getting better at editing, presentation, script writing and thumbnails. And the pivot test: am I still learning? If yes, keep going; if not, and the results aren't changing, it may be time to pivot. The figure does not say what was learned from this video; it shows the questions, not answers.

Claimmy argument, drawn from the Mario Effect and my own experienceBecause of the benchmark we mentally place on it, anything less is seen as deducting points from our overall score. So, what happens next?

The moment you quit

I know a few people who have had some success publishing content and it gaining traction. But their follow-up work fails to reach the same levels of success. Then they stop. The negative score has taken the fun and excitement out, and any new motivation is met with a feeling of wasting time.

LimitsThis rests on a handful of people I know and my own channel. I can’t say how common it is, or that the benchmark, rather than tiredness or other work, is why they stopped.I’ve experienced moments of this in the earlier years of my YouTube channel. Sometimes I would lose subscribers for every video I published. Not exactly the intended goal.

What you can do instead

The phrase “you either succeed, or you learn” comes to mind here. If something fails, then it’s a learning point. You should be able to identify the actions you can do differently next time to iterate on your winning formula.

So when considering a project, think of it in terms of what you can learn, rather than what success it can get. Your goal is to hit some measure of success (views, likes, watch time, subscribers, revenue), but your strategy is to implement specific things that might yield those results. You create an environment to learn.

Only when you have a winning formula can you focus less on changing strategy and more on consistency.

Be consistent with your actions

A competing explanationOutcome numbers aren’t only pressure. They’re also the feedback that tells you which actions work. Ignoring them entirely would leave you consistent but blind, so the point is how you feel about them, not whether you look.Do not look at it in terms of consistent success, but rather consistent action. Claima practical recommendation from experience, not testedYou cannot control the external measure of success. But you can control the actions you take. So rather than wanting your next YouTube video to have 10K views, instead focus on making one new video every month.

But you may have to adapt your strategy. Doing the same thing will not yield different results. As my dad would always say, “If you always do what you’ve always done, you’ll always get what you’ve always got.” Nothing changes.

What about timing?

Okay, there is a real possibility that we keep making changes, but what was really needed was consistency doing the same thing and waiting for that overnight success to happen.

The question that is always hard to answer is, at what point do you pivot your approach versus working for long enough for the results to show?

I would answer this with a simple question. Are you still learning? If you’re not learning and the results aren’t changing, then maybe you’re close to the point that you need to pivot.

What you can learn

There are two types of learning you can gain from your effort. First, the strategic learning: what inputs yield which outputs. For example, is it better to make a 3 minute or a 15 minute video? Is it better to just do voiceover, or a mixture of live camera and b-roll? What different types of video hooks can be tried?

The second is skills to learn. If you are improving your skills even though you’re not getting successful results, then you may consider it a worthwhile exercise. You may be improving your editing skills, presentation skills, script writing skills, thumbnail designs and so on.

What would change my mind?If people who keep learning on a project that isn’t working tend to do worse than those who stop early and try something else, the “still learning” test would be keeping people in the wrong place.But when you are no longer learning new skills, and you’re not testing new strategic ideas, then it’s time to pivot. Claima rule of thumb from my experienceJust make sure you give it enough time to predict when the results of your effort pay off. It might be days, it might be weeks.

The ah-ha moment

Pay attention to how you measure your success. If your results fall short of your measurement, this could have a demotivating effect causing you to quit. Instead, measure your actions and value the skills you learn. Treat things as an experiment to ensure you either succeed or learn.

Sources

  1. The Super Mario Effect: tricking your brain into learning more, Mark Rober, TEDxPenn, April 2018

Books behind this essay

Not sources for any claim here. These are books I have finished that shaped how I think about this question. Where I rely on evidence, it is cited in the text.

The whole shelf