Letter / word frequency changed in French?

[janmattijs]

Hi! Since a few weeks when playing the French localised version, possible words and scores have gone down dramatically. Most 4x4 grids now yield less than 100 words, and grids making several hundreds that you came across once in a while have all but disappeared. Seems related to letter frequencies. It has made playing more difficult (why not) but to the point of becoming a bit frustrating. Is this deliberate?

[pserwylo]

Thanks for the feedback. This game really depends on native speakers to provide such feedback due to the wide array of languages and the fact I am monolingual.

Is this deliberate?

Well, yes. That is to say, the change in letter probabilities was deliberate, but making it more difficult/frustrating was clearly not!

You can see #372 for a discussion of why we did this.

However, nobody wants the game to be less fun. So perhaps the easiest way to tackle it is to do what was discussed in the first recommendation of #359 - generate a few boards, and pick the one with the right number of words. Then when starting the game we can randomly generate 10 boards or so and pick the one with the most words. Though this can also be an issue for languages which tend to generate lots of words - we don't really want boards with 600 words on them. So the idea of a "target range" where we aim for, e.g. 200 words in a 4x4 board, more in a 5x5, etc may be the way to go.

Thoughts?

[janmattijs]

Hi Peter! Thanks for the explanation, indeed the change in game dynamics is simultaneous with the appearance of Qu. From a quick look, the changes you did after #372 make a lot of sense (shame on me, I hadn't noticed F never showed up!). A target number of words around 200 also makes sense. Indeed, I achieved my highest scores on boards with many words (say 3-400, sometimes more). But in those cases playing often became robotic, e.g. finding as many verbs with conjugations ending in -asse or -isse or -ates, which are highly unusual in actual language (I remember one with "merdâtes" that made me laugh out loud). However, this doesn't explain why the average word count has now dropped to around 70, because there are much less vowels. In most cases one side of the grid is completely blocked, no vowels. Long words (5 points or more) are now a rarity. One hypothesis: I play the no diacritics version. Could it be that when removing the accented vowels you made the vowels less frequent on "no diacritics" boards? I'll try to look closer into the discussions you point to above, maybe other thoughts will come.

[janmattijs]

Hi Peter, Have you changed something? The grids have become much more playable since the day before yesterday :-)

[pserwylo]

Haha, no changes yet, no. But I have started looking into it. Turns out that we already have code to continue generating up to 5 boards until at least 45 words are present.

I will improve this a little, and make it so that:

• We always generate 5 candidate boards (in the future maybe more depending on performance) and pick the best - right now it stops as soon as it hits some minimum criteria.
• Pick the board with the number of words that is closest to our desired target number - instead of always choosing that with the highest - boards with 1000 words would be no fun.
• Change the threshold from 45 to 200 - but make it configurable per-language in case some have dictionaries with more or less words.

This should end up with more playable boards. Then, in the future, it should be trivial to:

• Generate more than 5 candidate boards to get a more consistent experience.
• Use other heuristics to rank boards, e.g. those with letters that cannot be used should be ranked lower, even if the wordcount is spot on.
• Other... What else makes for a "good" board?

I'll post back here when done.