Closed snakeying closed 10 months ago
Bot detected the issue body's language is not English, translate it automatically.
Title: [Bug] Search engine is unstable
我在本地调试的,并且输出了百度爬到的内容,并没发现问题,可能是网络原因导致的。
Bot detected the issue body's language is not English, translate it automatically.
I debugged it locally and output the content crawled by Baidu. No problem was found. It may be caused by network reasons.
调试反馈,使用docker本地部署的话确实没有问题,看来应该是vercel限制了?
Bot detected the issue body's language is not English, translate it automatically.
Debugging feedback, there is indeed no problem if you use docker for local deployment. It seems that it is limited by vercel?
可以在日志中把插件中的输出打印一下看看具体原因,也不好说是不是百度限制了vercel的ip访问。
Bot detected the issue body's language is not English, translate it automatically.
You can print the output from the plug-in in the log to see the specific reason. It's hard to say whether Baidu has restricted vercel's IP access.
经过多次尝试,总结了一下搜索插件的情况
1.在vercel部署,不建议使用CHOOSE_SEARCH_ENGINE变量,因为baidu自动屏蔽vercel的IP导致返回302错误,google则表现不稳定,经常抽风
2.在私人服务器上部署(配合openwet passwall) 可以使用CHOOSE_SEARCH_ENGINE变量,无论是baidu还是google都可以,但搜索质量不敢恭维,还不如webchatgpt插件。
3.bing跟serp 的api搜索质量明显比直接使用搜索引擎要好,但严重受限,serp每月只有100条,bing每月1000条还有TPS限制。
建议加入Google Search Plugin,需要获取谷歌自定义搜索 API 密钥和谷歌自定义搜索引擎 ID,每天的查询数上限为 10,000 次。
在env中需要加入
GOOGLE_API_KEY="XXXXXXXX"
GOOGLE_CSE_ID="XXXXXXXXX"
相关网址
https://developers.google.com/custom-search/docs/tutorial/creatingcse
https://developers.google.com/custom-search/v1/introduction?hl=zh-cn
Bot detected the issue body's language is not English, translate it automatically.
After many attempts, I summarized the situation of searching for plug-ins.
When deploying in vercel, it is not recommended to use the CHOOSE_SEARCH_ENGINE variable, because Baidu automatically blocks vercel’s IP, causing a 302 error to be returned, and google’s performance is unstable and often convulses.
Deploy on a private server (with openwet passwall) You can use the CHOOSE_SEARCH_ENGINE variable, whether it is Baidu or Google, but the search quality is not flattering and is not as good as the webchatgpt plug-in.
The API search quality of Bing and SERP is obviously better than using the search engine directly, but it is severely limited. SERP only has 100 entries per month, and Bing has 1,000 entries per month and TPS limit.
It is recommended to join the Google Search Plugin and apply for Google's Custom Search JSON API. The maximum number of queries per day is 10,000.
https://developers.google.com/custom-search/v1/introduction?hl=zh-cn
经过多次尝试,总结了一下搜索插件的情况
1.在vercel部署,不建议使用CHOOSE_SEARCH_ENGINE变量,因为baidu自动屏蔽vercel的IP导致返回302错误,google则表现不稳定,经常抽风
2.在私人服务器上部署(配合openwet passwall) 可以使用CHOOSE_SEARCH_ENGINE变量,无论是baidu还是google都可以,但搜索质量不敢恭维,还不如webchatgpt插件。
3.bing跟serp 的api搜索质量明显比直接使用搜索引擎要好,但严重受限,serp每月只有100条,bing每月1000条还有TPS限制。
建议加入Google Search Plugin,需要获取谷歌自定义搜索 API 密钥和谷歌自定义搜索引擎 ID,每天的查询数上限为 10,000 次。
在env中需要加入
GOOGLE_API_KEY="XXXXXXXX" GOOGLE_CSE_ID="XXXXXXXXX"
相关网址
https://developers.google.com/custom-search/docs/tutorial/creatingcse
https://developers.google.com/custom-search/v1/introduction?hl=zh-cn
vercel 下的 CHOOSE_SEARCH_ENGINE 配置确实不稳定,该插件是基于爬虫实现的,本身就存在很多不稳定性。 该问题我会在文档中进行标注。
有考虑加入Google Search Plugin吗?σ^_^;
Bot detected the issue body's language is not English, translate it automatically.
Have you considered joining the Google Search Plugin? σ^_^;
有考虑加入Google Search Plugin吗?σ^_^;
我看 langchain 有这个插件,我试试引入进来。
Bot detected the issue body's language is not English, translate it automatically.
Have you considered joining the Google Search Plugin? σ^_^;
I see that langchain has this plug-in, I will try to introduce it.
Bot detected the issue body's language is not English, translate it automatically.
Then leave it to the boss ☆:.。. o(≧▽≦)o .。.:☆
在vercel部署
当CHOOSE_SEARCH_ENGINE变量为baidu时候
当不使用CHOOSE_SEARCH_ENGINE,默认DuckDuckGo时候则是正常的