search

alfiepoleon / kenya-news-scrapper

It scrapes various kenyan news sites and returns top news from each and a summary of each news item
Apache License 2.0
22 stars 9 forks source link

Avoid response body overhead #3

Closed danleyb2 closed 7 years ago

danleyb2 commented 7 years ago

https://github.com/alfiepoleon/kenya-news-scrapper/blob/d34c0f9a846c101862586d92be419df2e4e81635/scrape_web.py#L27-L27

i think, if you just checking the return status_code and connection availability, this is better suited

    request = requests.head(url)
alfiepoleon commented 7 years ago

Yes, didn't know that could be done, I will update that ASAP. http://docs.python-requests.org/en/master/api/#requests.get

alfiepoleon commented 7 years ago

Fixed