BBC_crawler does not work as expceted

Jellyfish042 / uncheatable_eval

Evaluating LLMs with Dynamic Data

MIT License

62 stars 3 forks source link

BBC_crawler does not work as expceted #3

Open jijivski opened 3 months ago

jijivski commented 3 months ago

It's nice to share these crawlers. When crawling BBC certain URLs, it returns None. I tried it on my PC and Kaggle environment as well, could you tell more about its environment?

Jellyfish042 commented 3 months ago

It seems to be an issue with the network environment. This script fetches CNN news URLs via Google News search. Please ensure your environment has unrestricted access to Google News search pages (as Google may block IPs with abnormal traffic), otherwise, it will not function properly.