dateparse can't handle cases where the same word has different meanings

[noviluni]

There are languages where some words have different meanings. This is generating some issues like this: https://github.com/scrapinghub/dateparser/issues/337

I'm creating this issue to track this and try to find a solution.

As far as I have seen, the translation is performed before the date is parsed, so we can't select the valid meaning using the other part as context. Apart from that, those words are inserted into a Python dictionary containing "word: meaning", overriding the other words with a different meaning.

Using regex simplifications could fix some use cases, but is not a valid approach for most of the cases.

In some cases (like in the word "mar" for Italian), we could detect when there are double elements (for example two months) and find if there is any duplicated key for any of them and try to find the real meaning, but this would be probably hard to implement.

Please, if you have any idea don't hesitate to comment on this issue.

---

List of cases:

• In Asturian (as), Spanish (es), Filipino (fil), Friulian (fur), Galician (gl), Italian (it), Greenlandic (kl), Mauritian Creole (mfe) and Albanian (sq): “mar” means “Tuesday” and “March”.
• The next languages use the same word for “week” and “Sunday”:
  • Embu (ebu): “kiumia”
  • Ewondo (ewo): "sɔ́ndɔ"
  • Filipino (fil) “linggo”
  • Nuosu (ii) “ꑭꆏ”
  • Georgian (ka) “კვირა”
  • Kikuyu (ki) "kiumia"
  • Bafia (ksf) “sɔ́ndǝ”
  • Indonesian (id): “minggu”
  • Ganda (lg) “sabbiiti”
  • Lao (lo) “ອາທິດ”
  • Meru (mer) “kiumia"
  • Macedonian (mk) “недела”
  • Kwasio (nmg) “sɔ́ndɔ”
  • Serbian (sr) “Недеља”
  • Yangben (yav) “sɔ́ndiɛ”

And there are a lot of other cases where the same word is used with different meanings:

• In Bulgarian (bg): “c” means “second” and “week”.
• In Bena (bez): “vil” means “monday” and “February”
• In Tibetan (bo): "ཟླ་བ་" means month and monday.
• In Breton (br): “meurzh” means “Tuesday” and “March”
• In Welsh (cy) “maw” means “Tuesday” and “March”
• In Taita (dav): “kaw” means “February” and “Tuesday”, “kad” means “Wednesday” and “March”, “kan” mean “April” and “Thursday” and “kas” means “Friday” and “May”.
• In Estonian (et): “t” means “hour” and “Tuesday”, “k” means “Wednesday” and “month”, and “p” means “day” and “Sunday”.
• In Gaelic (gd) “m” means “minute” and “am”
• In Hebrew (he) "שנ׳" means “minute” and “year”.
• In Hungarian (hu) “p” means “Friday” and “minute”.
• In Armenian (hy) “շաբաթ” means “Saturday” and “week”
• In Japanese (ja) “月” means “Monday” and “Month”, and "日" means “day” and “Sunday”.
• In Cape Verdean Creole (kea) “sig” means “second” and “Monday”.
• In Korean (ko) “월” means “Monday” and “month”, and “일” means “day” and “Sunday” .
• In Colognian (ksh) “m” means “minute” and “month” and “s” means “second” and “hour”.
• In Luba-Katanga (lu) “lum” means “Sunday” and “may”, and “lubingu” means “Saturday” and “week”.
• In Makhuwa-meetto (mgh) “sab” means “July” and “Sunday”.
• In Mundang (mua) “cla” means “February” and “Monday”.
• In Nuer (nus) “cäŋ” means “Sunday” and “day”, and “pay” means “month” and “July”.
• In Portuguese (pt) “seg” means “second” and “monday”.
• In Rundi (rn) and Kinyarwanda (rw) “kan” means “Thursday” and “August”.
• In Romanian (ro) “luni” means “month” and “Monday”, and “mar” means “Tuesday” and “March”.
• In Samburu (saq) “ong” means “April” and “Tuesday”, “ile” means “Thursday” and “June”, and “sap” means “July” and “Friday”.
• In Albanian (sq) “Sht” means “Saturday” and “September”.
• In Swedish (sv) “mån” means “Monday” and “Month”, and “m” means “minute” and “month”
• In Urdu (ur) “ہفتہ” means “Saturday” and “week”.

[noviluni]

There is a new case: in French "sept" means "September" but also "seven" and it gets confused.

Original issue: https://github.com/scrapinghub/dateparser/issues/819

[Gallaecio]

One raw approach I can think of is to translate all possible ways. Create as many copies of the translated strings as combinations of possible interpretations are possible, and yield a result for the first match possible.

However, instead of passing the first translation through all parsers and then trying the second translation, it may make sense to pass all translations through the first parser, then all translations through the second parser, etc.

Hopefully setting a language and a date order will allow to obtain the expected result in most cases.