fhamborg / news-please

news-please - an integrated web crawler and information extractor for news that just works
Apache License 2.0
2.08k stars 428 forks source link

AttributeError: 'module' object has no attribute 'request' #34

Closed gambolputty closed 7 years ago

gambolputty commented 7 years ago

Hello!

After a fresh install I ran the example code from the readme file and it gave me the following error:

>>> from newsplease import NewsPlease
>>> article = NewsPlease.from_url('https://www.nytimes.com/2017/02/23/us/politics/cpac-stephen-bannon-reince-priebus.html?hp')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/gregor/anaconda/lib/python2.7/site-packages/newsplease/__init__.py", line 79, in from_url
    articles = NewsPlease.from_urls([url])
  File "/Users/gregor/anaconda/lib/python2.7/site-packages/newsplease/__init__.py", line 98, in from_urls
    html = SimpleCrawler.fetch_url(url)
  File "/Users/gregor/anaconda/lib/python2.7/site-packages/newsplease/crawler/simple_crawler.py", line 16, in fetch_url
    return SimpleCrawler._fetch_url(url, False)
  File "/Users/gregor/anaconda/lib/python2.7/site-packages/newsplease/crawler/simple_crawler.py", line 27, in _fetch_url
    req = urllib.request.Request(url, None, headers)
AttributeError: 'module' object has no attribute 'request'

Python 2.7.13 (on macOS 10.12.5 Sierra) urllib is installed

Unfortunately I had no time for debugging. Might come back to it soon.

Gregor

fhamborg commented 7 years ago

Thanks for letting me know. The issue is that the function / module names are different in Py2.7

fhamborg commented 7 years ago

Alright, should be fixed now. You can update your package via pip install -U news-please

gambolputty commented 7 years ago

Awesome. Vielen Dank :)

jeradf commented 7 years ago

The same issue exists for commoncrawl_extractor.py.

fhamborg commented 7 years ago

can you open a new ticket with a detailed bug report for the problem that specifically occurred to you? otherwise, i will not be able to fix the issue.