Solving Wordle using information theory - wordle.plus

Solving Wordle using information theory

3Blue1Brown
Views: 1013594
Like: 43618
An excuse to teach a lesson on information theory and entropy.
Help fund future projects: ​
Special thanks to these supporters:
An equally valuable form of support is to simply share the videos.

Note, the way I wrote the rules for coloring while doing this project differs slightly from the real Wordle when it comes to multiple letters. For example, suppose in a word like “woody” the first ‘o’ is correct, hence green, then in the real Wordle that second ‘o’ would be grey, whereas the way I wrote things the rule as simply any letter which is in the word somewhere, but not in the right position, will be yellow.

To be honest, even after realizing this differed from the proper rule, I stuck with it because it made the computation of the full matrix of word-combination patterns more elegant (and faster), and the normal rule has always slightly bothered me. Of course, it doesn’t make any difference for the actual lesson here on entropy, which is the primary goal, and at least as I’ve gone back tried rerunning some of the models with the correct convention, it doesn’t really change the final results.

Contents:
0:00 – What is Wordle?
2:43 – Initial ideas
8:04 – Information theory basics
18:15 – Incorporating word frequencies
27:49 – Final performance

Original wordle site:

Music by Vincent Rubinetti.

Shannon and von Neumann artwork by Kurt Bruns.

Code for this video:

These animations are largely made using a custom python library, manim. See the FAQ comments here:

You can find code for specific videos and projects here:

——————

3blue1brown is a channel about animating math, in all senses of the word animate. And you know the drill with YouTube, if you want to stay posted on new videos, subscribe:

Various social media stuffs:
Website:
Twitter:
Reddit:
Instagram:
Patreon:
Facebook:

500 Comments

  1. I start with audio for the amount of vowels I can fit in one word. then reply and stink to get a lot more letters out of the way.

  2. Not sure how mathematically sound this is, but when I get a Green on first try, I wouldn't repeat it on the second. So that I get an extra slot for guessing another character, instead of reusing the Green which I definitely know it's correct already

  3. "…one in a thousand, because they're really obscure"
    highlights "Chara"

    The bot has not played Undertale.

  4. Anyone else who thought about this for 10 minutes and just used the word with the most common letters in the letter frequency (‘atone’ or something like that’)?

  5. Perfect explanation + visualization. Thank you!

  6. Hi, I am interested in learning information theory/entropy math you used for solving wordle. Can you please provide some pointers/links to reference material I can read and then incorporate those ideas in my own code to solve wordle? Thanks, I will really appreciate it!
    PS: I do have engineering/CS background so I should be able to read and understand the text.

  7. Balancing 'hitting' probability with information gain sounds like multi-armed bandit.

  8. So Wordle is basically Mastermind with extra pieces?

  9. I like using the word audio as it uses 4/5 vowels

  10. in terms of permutations
    for 5 letter words, there are 26^5 = ~ 11 mil words.
    for the complete grey word you rule out the possibility of 5 letters, which makes the options as 21^5 = ~ 4 mil words
    for every green match we reduce the power by 1, so 2 matches means 21^3 = ~9k words. i would then use human common sense to guess the most common and correctly sounding word

  11. I open with AIMER, have only had one round go to the fifth line since using it.

  12. Neumann sketch was amazing (although I think he was a fatter, once his wife told the only thing he can't calculate is calories), he is a personal hero of mine and probably the smartest man ever lived in terms of raw brainpower, and the number of subjects he touched. Anyone who is interested in anything related to numbers should know this giant of a mind. Just go to his wikipedia and look at what he is known for (apart from pure mathematics some examples are computer architecture, game theory, cellular automata, quantum mechanics, consultant at the atomic bomb etc.) and read about his childhood. It is impossible to not admire and feel inferior after knowing about such intellect. Obviously one should not feel worthless knowing about such people, because without this type of brain handful of others made greater contributions to mankind, yet it is amazing what the human mind is capable of even if 50% of the stories are true about von Neumann.

    I have sometimes wondered whether a brain like von Neumann's does not indicate a species superior to that of man. – Hans Bethe

    For a video request, I think cellular automata is a fascinating topic and 3B1B should definitely make a video about it. Complexity arising from simplicity is just an amazing concept, that makes one wonder what would be the rules of our universe if it was simulated with principles of automata.

  13. I worry that there's more 5 letter words in the English language that start with a W, end with a Y and have an R somewhere in them than weary, wordy, wormy, and wryly.

  14. As you may have noticed the entropy of the prior is not equal to the entropy of posterior + information gained, but they seem somewhat close. I wonder if there's any research into their relations.

  15. I'd be interested how much the expert mode restrains the information possibilities

  16. This 30 minute video taught me more about entropy than an entire section on entropy from my machine learning course. Bravo!

  17. COBOLISP I must remember nails and other.

  18. I like to open with "Knock"

    That's right internet. Come get me.

  19. Your code runs much faster than my code. Then I looked at `pattern_trit_generator`. With the original Wordle, if you guess HELLO, and if the answer is LIGHT, only the first L in the guess will turn yellow. Because my code runs (imo) too slowly on that specific part, I'm hoping to find something fast that generates the correct response.

  20. the wordle people saw this and made sure that CRANE yielded very little info on the next puzzle.

  21. But now I'm curious what happens if the second attempt doesn't try to get the word right yet but when the first two attempts only maximize the information you get. That would reduce the times you get it right in the first two attempts by a lot, but it would also reduce the amount of times get over four attempts.

    Considering you're always guessing at turn three I think this would perform significantly worse than your last used method, but intuitively, I would for the same reason not ever want to get it on turn three. I don't like winning on a guess that is at best ~50/50. I like learning if what I think are the most likely combinations are the ones that are more likely to give me enough information to actually know the answer by turn four and what the actually more informative combinations are for turn one and two.
    Then I would just always put in those two answers and works from there. That's the fun way to approach that puzzle for me. If the goal is to get it on turn three, I'll just flip a coin; I can do that anywhere anytime without breaking my head on words I never heard about in a list that excludes thousands of common words.

    *installs coinflip app that has various weight distributions of many common coins*
    I'm from the future, one where that app exists :p

  22. Watching this video while taking cs109 as a freshman at stanford and I’ve never been able to appreciate the math and how you presented it in one of your videos more than right now

  23. Smart trick there, Grant. Speeding up the playback rate to make it seem shorter that it really was. But even then, it ended up being over 30 minutes 😉 Nice video nevertheless, as always.

  24. What I would like to know is why didn't grant use a different word from CRANE?

    AUDIO, ADIEU or OURIE or LOUIE for example tell you more about the word IF you can remove as many vowels as possible.

    Especially when you consider the positions of these letters. Why use it's frequency when you can understand where a vowel is most likely to be?

  25. is there a way to develop this way of thinking ?

  26. I use STARE daily. I combine 3 letters from the wheel of fortune RSTLNE and knock out 2 vowels.

  27. Honestly, I expected Grant's starter word to be "PENIS".

  28. Video seemed interesting but I found his voice very grating unfortunately

  29. The wordplay at 18:40 gave me so much joy, totally caught me off guard in the best way

  30. Disappointed you didn't use a state of the art reinforcement algorithm with a (very) deep neural network for this 😂 just kidding, great video !

  31. I love ur content but please please install dark reader plugin. The glaring white screens make me click away every time.

  32. I've always used letter frequency to test vowels as quickly as possible, so my first guess is a word with at least three vowels. Then I always try to put what I learned in the first guess to use in the second guess, because why make two blind guesses?

    Anyway. Information theory is nice but I'm surprised that this algorithm doesn't consider that there are only six vowels in the English language, and they're critical for making English words.

  33. I like that it's not too different than. Est first letters in wheel of fortune. RSTLNE

  34. New York Times be like "aigh't imma buy this channel as well"

  35. Wordle seems like an old Total Drama Island game where Cody needs to find some numbers to dezactivate a bomb from a trash can. The game was from an old Total Drama Island website.

  36. What about adding in a factor considering previous games answers and how likely they will be reused? That should narrow it down a lot

  37. Genius!!!! I mean how do people guess in less than 3 shots!

  38. Why is this keep getting recommended to me? I have no interest in programming and I also never heard of or played this game (if it's even a game, didn't watch the video).

  39. I'd be more interested to see how you'd programme it playing hard mode

Leave a Reply

Your email address will not be published. Required fields are marked *