damus-io / damus

iOS nostr client
GNU General Public License v3.0
1.96k stars 288 forks source link

detect chinese dialect #2270

Open alltheseas opened 1 month ago

alltheseas commented 1 month ago

potential approach:

-detect correctly -dont send language detect

alltheseas commented 1 month ago

@tyiu do you know how apple detects Chinese language(s)?

tyiu commented 1 month ago

Can you provide more context? Is there a specific example? Which Chinese dialects are you referring to? In general, I don't think what you're asking is even possible.

jb55 commented 1 month ago

maybe this got lost in the meeting notes, but feedback from @semisol was:

jb55: Hmm these don’t seem to translate for some reason, not supported? https://damus.io/nevent1qqst492mjessnswr6usw0007kff977ka4s8zpjlatnlvrsr3jlhmwtcpzpmhxue69uhkummnw3ezuamfdejsz9nhwden5te0wfjkccte9ehx7um5wghxyctwvsq32amnwvaz7tm9v3jkutnwdaehgu3wd3skueqpz3mhxue69uhhyetvv9ujuerpd46hxtnfduncq76h

semisol: oh yeah, I you need to specify zh- (replace with something I forgot) not zh-

Semisol commented 1 month ago

you need to specify zh-Hant/zh-Hans instead of zh if you are doing that. NLLanguage does that for you actually, so I don't know what's happening.

Semisol commented 1 month ago

It works for me. I'm also receiving requests for translating Chinese, but I only have received <50 in the last 7 days.

image
tyiu commented 1 month ago

you need to specify zh-Hant/zh-Hans instead of zh if you are doing that. NLLanguage does that for you actually, so I don't know what's happening.

I added code to intentionally drop the variant part of the detected locale from NLLanguageRecognizer. I didn't realize that we shouldn't be doing this for Chinese. https://github.com/damus-io/damus/blob/c99aaea5980121a89c7499665fcb5d45faa1f936/nostrdb/NdbNote.swift#L475-L477

tyiu commented 1 month ago

For example, DeepL expects only zh as the source locale. So maybe it depends on the translation service.

https://developers.deepl.com/docs/resources/supported-languages#source-languages

Semisol commented 1 month ago

For example, DeepL expects only zh as the source locale. So maybe it depends on the translation service.

https://developers.deepl.com/docs/resources/supported-languages#source-languages

Yeah. I think for Damus Purple the locale should be dropped at the API layer, not client.