Update Dec 25 2023 with 2023 scores. Hade a great time again, thanks Eric and Reddit!
Since 2018 I’m participating in 🎅Advent of Code🎅 and enjoying it a great deal. AoC has been running since 2015, so a sizable amount of data has been generated. Let’s see what we can learn, starting with the amount of stars awarded.
In total there have now been more than 18M stars awarded (+4M compared to last year)! And 2023 is just barely finished, many people will earn stars in the days to come.
Edit: as BBQspaceflight indicated on the AoC Reddit, probably 2016 was not a rough year, but many people have been solving 2015 at a later time (e.g. they participated in 2018 and afterwards did 2015).
For a better looking plot, look at Maurits vd Schee. It’s a familiar sight: in general the times are below the hour mark, with a couple going higher. After 2020, the completion times are more compressed. 2023 most difficult puzzles were Step Counter and Never Tell Me The Odds at 24th spot with 62 minutes. Personally I spend a whopping total of nearly 20 hours on these two puzzles, almost 50% of my total time this year.
Low completion times can be a result of two factors:
- The puzzles were easier
- The participants where better / more competitive
One way of investigating the difficulty of a year is by analyzing the completion rate: how many people got all the stars compared to the people that got only 1 star of day 25. These people did make it to day 25, thus put a considerable amount of effort in, but couldn’t finish all puzzles.
In the above chart, each rectangle symbolizes the people that solved all puzzles during the year. The height shows the completion rate.
The completion rate was very high in 2016 and 2017 and has dropped in recent years, corresponding with more participants each year.
For 2023 the verdict is still out, right now day 25 has just opened up, so relatively more hardcore participants have finished it, leading to a high completion rate. I checked about 1 hour day 25 became available the completion rate was even 90%.
df| year | both | only_first | perc_day_25_both | x1 | x0 | |
|---|---|---|---|---|---|---|
| 0 | 2015 | 6524 | 2502 | 72.0 | 6524 | 0 |
| 1 | 2016 | 4130 | 580 | 87.0 | 10654 | 6524 |
| 2 | 2017 | 6746 | 1167 | 85.0 | 17400 | 10654 |
| 3 | 2018 | 3965 | 1766 | 69.0 | 21365 | 17400 |
| 4 | 2019 | 4110 | 1142 | 78.0 | 25475 | 21365 |
| 5 | 2020 | 14198 | 4607 | 75.0 | 39673 | 25475 |
| 6 | 2021 | 11940 | 6243 | 65.0 | 51613 | 39673 |
| 7 | 2022 | 13093 | 7469 | 63.0 | 64706 | 51613 |
| 8 | 2023 | 4097 | 1105 | 78.0 | 68803 | 64706 |
Again note that 2023 is very fresh still.
We see that:
- The amount of finishers (people getting all stars) went up in 2020
- The amount of people that got points on the leaderboard is slowly increasing. Probably this is an indication of more competion.
- The percentage of finishers getting points is varying.
Another indicator can be the time it took to solve a puzzle.
The fastest completion times add up to around 3 hours, which is amazing. Since nobody ever finished #1 at all puzzles, this is a theoretical minimum.
The completion times of #100 add up to a more ‘human’ amount. These times are still way below the amount of time a ‘normal’ participant spends on AoC. For example I consider myself an enthusiast, but my completion times are normally about 2-3x the #100. Day 21 though this year…. 12 hours….
2023 was similar compared to 2021 and 2022.
2023 will move more to the right given time. I’m not sure anymore if this graph is indicative of anything.
Getting leaderboard points is special (I made it once for the first time ever this year, yay!). There are people who do it consistently. Let’s give the top 30 some extra recognition🎈
All the people on this top 30 list are amazing, but some awards to hand out:
- 🏆Robert Xiao managed to get the most amount of points and overall most leaderboard placements
- 🏆betaveros got on average most points & leaderboard entries (ignoring anonymous user here). betaveros also managed to get 50 entries is 2018, which was a one-time event
- 🏆glguy for getting the highest score while getting points in all 9 seasons
Doing AoC once and get LB points is nice, but it’s even nicer to do it twice, thrice, etc.
Most of the people that get points manage to do it only once. The y-axis is logaritmic. Who are having so much grit to get points all 9 seasons?
| user | amount_seasons | total_points | total_lb_placements | |
|---|---|---|---|---|
| 11 | glguy | 9 | 12622 | 202 |
| 16 | etotheipi1 | 9 | 11108 | 204 |
| 20 | msullivan | 9 | 9951 | 173 |
| 24 | Kevin Yap | 9 | 9224 | 164 |
| 43 | (anonymous user #60233) | 9 | 6651 | 131 |
We lost Roderic Day and lukechampine compared to last year. glguy is topping the list. Coming back to our competitiveness discussion, how many points did they score together?
It’s varying but since 2020 the points are. This could reflect:
- increased competitiveness during the years
- natural variation
- legends getting older😊
All in all an amazing achievement! All in all I think there is a strong case for AoC being more competitive after 2019.
Let’s finally turn to which puzzles were easiest or hardest.
| puzzle | user | time (seconds) | |
|---|---|---|---|
| 0 | 2022-3-1 | ostwilkens | 10 |
| 1 | 2023-1-1 | (anonymous user #640116) | 12 |
| 2 | 2022-4-1 | max-sixty | 16 |
| 3 | 2019-1-1 | bluepichu | 23 |
| 4 | 2018-1-1 | Tris Emmy Wilson | 26 |
Obviously LLMs are playing a role here
| puzzle | lb full (seconds) | |
|---|---|---|
| 0 | 2021-1-1 | 65 |
| 1 | 2022-1-1 | 76 |
| 2 | 2019-1-1 | 84 |
| 3 | 2018-1-1 | 92 |
| 4 | 2021-2-1 | 98 |
The leaderboard capped (the #100 completed the puzzle) after barely a minute in 2021 for the first star!
| puzzle | user | time (minutes) | title | |
|---|---|---|---|---|
| 449 | 2018-15-2 | Simon Parent | 36 | Beverage Bandits |
| 448 | 2018-17-2 | Raven Black | 33 | Reservoir Research |
| 445 | 2018-24-2 | Simon Parent | 28 | Immune System Simulator 20XX |
| 444 | 2022-22-2 | mrphlip | 25 | Monkey Map |
| 443 | 2020-20-2 | xiaowuc1 | 25 | Jurassic Jigsaw |
| 442 | 2022-19-2 | lukechampine | 24 | Not Enough Minerals |
| 440 | 2021-23-2 | goffrie | 23 | Amphipod |
| 439 | 2019-18-2 | glguy | 22 | Many-Worlds Interpretation |
| 438 | 2015-22-2 | Paul Hankin | 21 | Wizard Simulator 20XX |
| 437 | 2021-19-2 | ecnerwala | 21 | Beacon Scanner |
The longest 3 solve times were all in 2018! Shoutout to Simon Parent for solving 2 out of the top 3. This list mostly has puzzles that just take a long time to code, with Beverage Bandits as perfect example.
| puzzle | lb full (minutes) | title | |
|---|---|---|---|
| 449 | 2015-19-2 | 232 | Medicine for Rudolph |
| 448 | 2015-1-2 | 186 | Not Quite Lisp |
| 447 | 2015-22-2 | 183 | Wizard Simulator 20XX |
| 446 | 2016-11-2 | 164 | Radioisotope Thermoelectric Generators |
| 443 | 2018-15-2 | 143 | Beverage Bandits |
| 442 | 2019-22-2 | 123 | Slam Shuffle |
| 440 | 2019-18-2 | 117 | Many-Worlds Interpretation |
| 438 | 2018-23-2 | 100 | Experimental Emergency Teleportation |
| 437 | 2016-22-2 | 88 | Grid Computing |
| 436 | 2018-24-2 | 87 | Immune System Simulator 20XX |
If we look at when the leaderboard capped some different puzzles show up. I feel that this list has some more algoritmic challenges (Slam Shuffle for example, but Medicine for Rudolph as well).
2023 does not show up in the top 10! We find day 21 Step Counter at 13th spot with 76 minutes and day 24 Never Tell Me The Odds at 24th spot with 62 minutes.
Overall, I feel 2018 is a strong contender for the most difficult year, although with increased competitiveness it’s getting more difficult to compare it to the recent years.
Hope you enjoyed this analysis and see you back next year! 🎄⭐🎅