unitedstates / congress

Public domain data collectors for the work of Congress, including legislation, amendments, and votes.
https://github.com/unitedstates/congress/wiki
Creative Commons Zero v1.0 Universal
914 stars 198 forks source link

TypeError: urlopen() got an unexpected keyword argument 'timeout' #152

Open geduldig opened 9 years ago

geduldig commented 9 years ago

I'm running on Python 2.7. Looks like my urllib2 is not compatible. Do I need a different version of urllib2 than what comes with Python 2.7?

konklone commented 9 years ago

Can you do a pip freeze and show us the version you're using so we can try it out? Also, can you show us the error and stack trace you see, and what command you ran to generate it?

Also FWIW, we still have the desire to move to Python 3.X. A previous effort stalled, and needs to start from scratch (not that it's that hard).

geduldig commented 9 years ago

I discovered the problem. Some dependencies were the wrong version. I just realized I could run pip install -r requirements.txt, and that fixed everything.

I would be very interested in support for Python 3.

nathancmo commented 6 years ago

I have run into the same issue. I ran pip install -r requirements.txt and had to do a couple separately. I'm running python 2.7.

The following is the attempted run as well as the stack trace.

Nathans-Air:congress-master Nathan$ ./run votes --congress=112 --session=2012
Traceback (most recent call last):

  File "./run", line 71, in <module>
    task_mod.run(options)

  File "/congress-master/tasks/votes.py", line 41, in run
    to_fetch = (vote_ids_for_house(congress, session_year, options) or []) + (vote_ids_for_senate(congress, session_year, options) or [])

  File "/congress-master/tasks/votes.py", line 75, in vote_ids_for_house
    options)

  File "/congress-master/tasks/utils.py", line 315, in download
    response = scraper.urlopen(url, **urlopen_kwargs)

TypeError: urlopen() got an unexpected keyword argument 'timeout'

This error also happens when trying to run ./run votes [--force [--fast] ]

I believe I have all the requirements from the requirements.txt file. Here is the full pip freeze output.

Nathans-Air:congress-master Nathan$ pip freeze
altgraph==0.10.2
appnope==0.1.0
asn1crypto==0.23.0
backports.shutil-get-terminal-size==1.0.0
bdist-mpkg==0.5.0
beautifulsoup4==4.6.0
bonjour-py==0.3
certifi==2017.7.27.1
cffi==1.11.2
chardet==3.0.4
cryptography==2.1.1
cssselect==1.0.1
decorator==4.1.2
enum34==1.1.6
funcsigs==1.0.2
html5lib==0.999999999
idna==2.6
ipaddress==1.0.18
ipython==5.5.0
ipython-genutils==0.2.0
iso8601==0.1.4
lxml==4.1.0
macholib==1.5.1
matplotlib==1.3.1
mechanize==0.3.6
mock==2.0.0
modulegraph==0.10.4
numpy==1.8.0rc1
pathlib2==2.3.0
pbr==3.1.1
pexpect==4.2.1
pickleshare==0.7.4
prompt-toolkit==1.0.15
ptyprocess==0.5.2
py2app==0.7.3
pycparser==2.18
Pygments==2.2.0
pyobjc-core==2.5.1
pyobjc-framework-Accounts==2.5.1
pyobjc-framework-AddressBook==2.5.1
pyobjc-framework-AppleScriptKit==2.5.1
pyobjc-framework-AppleScriptObjC==2.5.1
pyobjc-framework-Automator==2.5.1
pyobjc-framework-CFNetwork==2.5.1
pyobjc-framework-Cocoa==2.5.1
pyobjc-framework-Collaboration==2.5.1
pyobjc-framework-CoreData==2.5.1
pyobjc-framework-CoreLocation==2.5.1
pyobjc-framework-CoreText==2.5.1
pyobjc-framework-DictionaryServices==2.5.1
pyobjc-framework-EventKit==2.5.1
pyobjc-framework-ExceptionHandling==2.5.1
pyobjc-framework-FSEvents==2.5.1
pyobjc-framework-InputMethodKit==2.5.1
pyobjc-framework-InstallerPlugins==2.5.1
pyobjc-framework-InstantMessage==2.5.1
pyobjc-framework-LatentSemanticMapping==2.5.1
pyobjc-framework-LaunchServices==2.5.1
pyobjc-framework-Message==2.5.1
pyobjc-framework-OpenDirectory==2.5.1
pyobjc-framework-PreferencePanes==2.5.1
pyobjc-framework-PubSub==2.5.1
pyobjc-framework-QTKit==2.5.1
pyobjc-framework-Quartz==2.5.1
pyobjc-framework-ScreenSaver==2.5.1
pyobjc-framework-ScriptingBridge==2.5.1
pyobjc-framework-SearchKit==2.5.1
pyobjc-framework-ServiceManagement==2.5.1
pyobjc-framework-Social==2.5.1
pyobjc-framework-SyncServices==2.5.1
pyobjc-framework-SystemConfiguration==2.5.1
pyobjc-framework-WebKit==2.5.1
pyOpenSSL==0.13.1
pyparsing==2.0.1
python-dateutil==1.5
pytz==2013.7
PyYAML==3.12
requests==2.18.4
scandir==1.6
scipy==0.13.0b1
scrapelib==0.10.1
simplegeneric==0.8.1
six==1.4.1
traitlets==4.3.2
urllib3==1.22
virtualenv==15.1.0
wcwidth==0.1.7
webencodings==0.5.1
xattr==0.6.4
xmltodict==0.11.0
zope.interface==4.1.1
JoshData commented 6 years ago

I'm running scrapelib version 0.10.0. It's possible version 0.10.1 isn't backwards compatible and that we need to upgrade our scripts (in fact, we should go to the 1.x releases.)

nathancmo commented 6 years ago

I uninstalled scrapelib 0.10.1 and installed scrapelib 0.10.0 and am still having the same issue.

nathancmo commented 6 years ago

After testing I got an error saying I was missing the module flask. Once installing this and opening a new terminal I am able to get data for the house.