search

leptonai / search_with_lepton

Building a quick conversation-based search demo with Lepton AI.
https://search.lepton.run
Apache License 2.0
7.73k stars 980 forks source link

How to answer search results in Chinese! #73

Open tc0715888 opened 5 months ago

tc0715888 commented 5 months ago

How to answer search results in Chinese!

ng-fukgin commented 5 months ago

你在后台把{"role": "user", "content": query},改成{"role": "user", "content": query+"请把你的回答翻译成中文"},很大几率会回答中文

hejxiang commented 4 months ago

Query by chinese and use a chinese model as the backend llm, like qwen, then the default output will be chinese

tc0715888 commented 4 months ago

How to set the backend as "qwen", can you provide guidance?

ng-fukgin commented 4 months ago

How to set the backend as "qwen", can you provide guidance?

你先自己用fastchat也好，什么也好，自己部署一个类似于openai 的api 的接口，然后修改 base_url=f"https://{self.model}.lepton.run/api/v1/",这一个链接，例如base_url=f"https:localhost:8000/v1/", 然后修改"LLM_MODEL": "mixtral-8x7b",这里的模型名字，就可以了，qwen不会生成Related，可能会报错，但不影响模型的回答。