mentalisttraceur
commented
3 years ago I thought of something similar before, because I also faced the problem of "how do I make it maximally easy for myself to manually transcribe arbitrary data", so when I saw this I thought it was really cool that someone else had gone through the same process!
I never actually coded my version, but in my idea I was using the wordlist used by the pwqgen utility (from Openwall's passwdqc). I don't know if you considered that wordlist, but I think you might find that word list more useful for your goals, because it has a few useful properties:
- All the words are 3-6 characters, so they are quick and easy to type even without autocomplete.
- The words are relatively distinct from each other, so it should still be easy to not lose your place.
- There are 4096 words, so you can encode 3 bytes into two words.
In my head I was initially calling the idea "base4096 encoding", just like you called yours "base256". Then to emphasize the human-friendliness and deemphasize any implication of space-efficiency, I decided to call it "human4096 encoding". Then I realized that was pretty English-centric of me, so I'm tempted to call it "english4096". That's nice because it also leaves room for many other "{{ human language name }}4096" encodings. Or maybe some other number than 4096. But 4096 seems like a good sweet spot for distinct words that humans can keep track of which also track byte boundaries with a reasonably regular period.
Biggest differences from your implementation:
- Biggest relative advantage of the english4096 encoding is that the encodings tend to be much smaller.
- english4096 is worse in cases where the auto-complete really dominates, like thumb input through a touchscreen keyboard on a phone.
- english4096 is better when using a hardware keyboard with physical buttons, or when auto-complete-like assistance isn't available.
-
Biggest relative disadvantage of the english4096 encoding is the implementation complexity: the lack of 1-1 mapping to bytes.
(12-bit steps over 8-bit bytes is regular enough that we can still do a simple loop which reads three bytes at a time and has a special case for the two possible short reads. But it's still more code and bit-twiddling than just a simple table lookup.
Also, english4096 needs one extra word to handle those two cases in the decoder - if this modifier word is present, the word it modifies only encodes the bits of the first of the two bytes it would normally encode bits of. My intuition is to put it in front of the last word rather than behind, to make the logic simpler in the decoder, since that way the decoder doesn't have to "hold back" the bytes for a word until after checking if the next word is a modifier. You could use a modifier character or capitalization to signal this instead of an extra word, but I think capitalization is worse for human-friendliness because it complicates the mental model from the stupidly simple "just type the words you see" with extra details like "you need to be mindful of capitalization/symbols". To someone with attention to detail and computer savvy, the idea of replicating text exactly is automatic, but to a lot of lay people it is not necessarily intuitive that special characters or capitalization can't be just ignored when transcribing.)
Anyway, you've inspired me to actually implement and document english4096 encoding! Cheers!
ctsrc
I really like the thought process behind this. Nice idea.