dhammacakka / pm12e

2 stars 0 forks source link

New Word Disher #31

Open bksubhuti opened 3 years ago

bksubhuti commented 3 years ago

Background. We could do a word frequency priority for our dictionary, but the word parts repeat and the knowledge for the first part can be recycled and the worker can go faster this way. It takes longer to get a usable product but it is faster.

We could go by word frequency, or by word frequency AND words not found in other dictionaries cped and ped. These are more valuable.

Solution: Give words based on word frequency .. use that as a base word.. then give out words until it stops repeating (sandhi parts). Then jump to the new word.. give more words until it stops repeating.