search

michaelthwan / searchGPT

Grounded search engine (i.e. with source reference) based on LLM / ChatGPT / OpenAI API. It supports web search, file content search etc.
MIT License
600 stars 62 forks source link

feat: Add multi-language support #83

Closed 1M4nt0 closed 1 year ago

1M4nt0 commented 1 year ago

Added multi-language support for localizing Bing searches and results from LLMService

michaelthwan commented 1 year ago

@1M4nt0 Thanks. I will have a look later.

michaelthwan commented 1 year ago

@1M4nt0 At first I thought your change is related to multi-language support for the website. Then I realize that you used a lib to translate strings in the backend.

However, I tried that, you only need to give instruction to the LLM to achieve the same result. Localizing languages, like prompt, is not required.

For example, this used master branch and only changed it-IT for bing search and added the language instruction in prompt. It can still control LLM response. image

Example2: I used added the language instruction zh (Chinese). Even for bing search = it-IT, it can still output the result with Chinese successfully. image

It is not magic but how the embedding work. Machine will translate prompt from whatever language into embedding, which preserve non-language-dependent semantic meaning. Therefore above example works.

Thus, I will not merge your later commits but only the self.sender NoneType issue. And I will make the bing/prompt language configurable (in config.yaml and UI). It will save people's localization work while making the project simpler.

Thank you.

1M4nt0 commented 1 year ago

Great! I found the way more complicated. Your solution is much better.