JMdictProject
commented
2 years ago With 80% of the entries basically being transliterations of the kana forms, I'm not sure tagging them as such would help a lot.
I've been aware from its inception that the raw JMnedict/enamdict data was a problem for downstream applications as some of the forms, e.g. 田中, have a heap of entries with different readings and transliterations associated with them. Simply bundling them was of limited help as some are relatively rare. To handle this in WWWJDIC I use a heavily-massaged version of enamdict in which:
- all the entries that share a common kanji form are merged into a single entry;
- a file of about 80k high-frequency kanji/kana combinations is used to enable the move of the more common forms to the front of the combined entries.
Thus for example, if you look up 田中 in WWWJDIC you get: 田中 【たなか】 (p,s,f) Tanaka 【でんちゅう】 (s,g) Denchuu 【たなた】 (s) Tanata 【たんか】 (s) Tanka 【だなか】 (s) Danaka 【なたか】 (s) Nataka 【ぬなか】 (s) Nunaka 【のなか】 (s) Nonaka 【ひろか】 (s) Hiroka 【やなか】 (s) Yanaka たなか and でんちゅう were moved to the front and the rest follow in 五十音 order.
This is all done as a daily post-processing exercise and doesn't require anything different in JMnedict itself.
Perhaps something like this is the way to go, rather than add extra tags to JMnedict? Having a [transl] or whatever tag would not be a lot of use in many cases. For WWWJDIC I'd probably filter it out.
stephenmk
Similarly to how we currently have [expl], [fig], etc., gloss types in JMdict, I am wondering if it would be both desirable and technically feasible to have a transliteration / romanization gloss type for JMnedict.
A common complaint among users of the Yomichan browser extension is that JMnedict entries often clutter up search results with low quality information. Some users eventually disable JMnedict entirely due to this annoyance. The source of the problem is that there are so many generic name entries with glossaries which only contain romanized forms of particular readings.
I put together a new version of JMnedict for Yomichan which consolidates much of this generic information. So far I've only received positive feedback. My program for producing this dictionary discards any glosses that it determines to be romanizations and then merges the related headwords and readings into single entries.
Search results for 田中
Search results for 樹下 compared with WWWJDIC
Search results for 岩波国語辞典 (since there's more info than just a romanization, these glosses are not discarded)
Would there be any interest in having this metadata stored directly in JMnedict in the form of glossary types or miscellaneous sense tags? If so, I can produce a list of sequence numbers, sense numbers, and glosses which I have determined to be transliterations.
By my estimate, over 80% of the glosses in JMnedict are transliterations. A lot of this information, particularly surnames and given names, does not seem to me to be particularly useful. For example, 田中 returns 10 surname readings, but there is no indication that it is almost always read as たなか.
However, I don't think anyone can deny that the other ~20% of the corpus is quite useful. There's been some discussion in the past about how JMnedict is sometimes overlooked by developers. Having this metadata might be one step we can take towards making the dictionary more appealing.