LaurensWeyn
commented
7 years ago Perhaps some combination of both is possible?
When the user sets a definition as preferred, both the dictionary form and the conjugated form of the preferred word could be stored. If it shows up conjugated the same way, then clearly they want the definition they chose the last time it showed up. If a word shows up in a different form though, using the preferred definition is at least a good guess as to the best definition.
But if this guess is wrong and the user decides a new preferred definition, we'll need one more piece of information. Will this be the definition for this form only, or the global default?
- If that definition is set as an exception, it will only be prioritized when it appears again in that form, and the 'old' preferred definition will still be used if it shows up in a new form.
- If the definition is set as a global default, the 'old' preferred definition will be assumed to only be used when the word is conjugated as it was, and both the conjugated and dictionary form for the 'new' preferred definition is set.
- A third option, override all, might also be present on a conflict to erase all the other preferred definitions, in the event that the user installs a new dictionary or something and wants to use that without going to every word in every form they've seen it in to reset the default.
I haven't thought this through thoroughly, but it seems like a good compromise between accuracy and convenience. It will require a new file format of course, but since preferred definitions are broken due to the hashing problem anyway this isn't a big deal.
wareya
Right now they're based on the dictionary form. This makes it a lot faster to set up preferred definitions since you don't have to set it on every conjugation of a given verb you run into, but there are situations where multiple preferred words can end up showing up for the same surface form because of deconjugation. Deconjugation is the only place that the surface form and dictionary form differ, so it seems like an inherent problem to me.
Maybe spark reader could get away with marking definitions as "good" instead of "preferred", so you would have multiple preferred definitions in cases like this, and it wouldn't care about the specific word the definition pops up on. This would also let users basically pull all the definitions they like for a given word to the top of the list, and if you added a "not good" thing too, they could also push definitions they don't like to the bottom. And maybe, just maybe, you could also have actual preferences for specific surface forms, on top of the "good definition / bad definition" thing, which would care about the specific word again.
The issue here is that doing what I described in the above paragraph would make it very complicated to use spark reader effectively, which is why I'm asking about this issue as a question instead of a request.
This is only tangential/unrelated to the blacklist, which is to prevent parsing mistakes like はそう being a single segment, and seems to work well. But the blacklist basically has to be based on the surface form only because that's what it's for, and it feels gross for the blacklist and preferred definitions to use a different version of the same word to determine whether it counts.
For what it's worth, I'm already using my branch with preferred definitions based on the surface form instead of the dictionary form, and it seems to work well, it's just tedious. Being able to mark specific definitions as "good" without caring about the word itself would make it less tedious, but more complicated.
One interesting note is that basing preferred definitions on surface form instead of dictionary form makes preferred definitions work better with changes to the deconjugator since you wouldn't run into a specific possible rare issue: If you mark two definitions as preferred for two different given dictionary forms, then the deconjugator changes and puts both definitions on the same surface form, and that surface form deconjugates to both dictionary forms, it's basically arbitrary which definition shows up first in the list.
This is basically a special case of the "multiple preferred definitions for the same conjugated word" issue, except that it means the behavior of the program changes under the user's feet instead of behaving differently for a word that they didn't set a preferred definition for yet. And if you set a specific one of those definitions as preferred, what happens to the other one? It was valid for its dictionary form, just for a different word. Is it no longer preferred?