search

stanford-oval / storm

An LLM-powered knowledge curation system that researches a topic and generates a full-length report with citations.
http://storm.genie.stanford.edu
MIT License
13.27k stars 1.21k forks source link

Connection timeout when running examples #88

Open satzynu opened 3 months ago

satzynu commented 3 months ago

Get all lot of ConnectTimeout errors when running examples via cli. example : 

Error while requesting URL('https://.......') - ConnectTimeout('timed out')
satzynu commented 3 months ago

Digging deeper ...

adding the user agent as shown below, to the httpx_client call in utils.py > WebPageHelper class, helped relieved the errors.

class WebPageHelper: .... headers = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36"} self.httpx_client = httpx.Client(verify=False, headers=headers)

I am not sure if this is the best header or if more headers need to added for optimal results. Hope it helps others facing this.