mcm
commented
8 years ago @tcw3bb I just tested this with requests 2.2.1 and it worked for me... would I be right in guessing that you have requests installed from your distro's repositories rather than from pip? I'm finding a lot of notes that various Linux distros unbundle the stuff in requests.packages for various distro-y reasons. Newer versions of requests have since worked around this issue, but 2.2.1 is fairly old and I suspect you'd have to upgrade your distro to get a newer pre-packaged version.
My best suggestion if upgrading the system version of requests isn't an option would be to run machinae out of a virtualenv.
Please let me know if my assumptions here are wrong and we can continue to work on troubleshooting the issue.
[gh_machinae_18] ~/t/gh_machinae_18 ❯❯❯ pip3 install 'requests==2.2.1'
You are using pip version 6.0.8, however version 8.1.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Collecting requests==2.2.1
Downloading requests-2.2.1-py2.py3-none-any.whl (625kB)
100% |################################| 626kB 886kB/s
Installing collected packages: requests
Successfully installed requests-2.2.1
[gh_machinae_18] ~/t/gh_machinae_18 ❯❯❯ python3
Python 3.4.3 (default, Feb 27 2016, 19:17:14)
[GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from requests.packages.urllib3 import exceptions
>>>
tcw3bb
root@tw-linux:~/Downloads# python3 /usr/local/bin/machinae -c /etc/machinae.yml -o N 184.105.247.196 Traceback (most recent call last): File "/usr/local/bin/machinae", line 9, in
load_entry_point('machinae==1.3.4', 'console_scripts', 'machinae')()
File "/usr/local/lib/python3.4/dist-packages/machinae/cmd.py", line 183, in main
cmd.run()
File "/usr/local/lib/python3.4/dist-packages/machinae/cmd.py", line 167, in run
output = outputs.MachinaeOutput.get_formatter(fmt).run(self.results)
File "/usr/local/lib/python3.4/dist-packages/machinae/outputs.py", line 40, in run
for row in result_sets:
File "/usr/local/lib/python3.4/dist-packages/machinae/cmd.py", line 98, in results
scraper = Site.from_conf(site_conf) # , verbose=self.verbose)
File "/usr/local/lib/python3.4/dist-packages/machinae/sites/init.py", line 46, in from_conf
from . import csv, html, rss, json, ipwhois
File "/usr/local/lib/python3.4/dist-packages/machinae/sites/csv.py", line 7, in
from .base import HttpSite
File "/usr/local/lib/python3.4/dist-packages/machinae/sites/base.py", line 14, in
from requests.packages.urllib3 import exceptions
ImportError: No module named 'requests.packages'
root@tw-linux:~/Downloads# pip3 list requests (2.2.1) urllib3 (1.7.1)
Changed line 14 in /usr/local/lib/python3.4/dist-packages/machinae/sites/base.py from requests.packages.urllib3 import exceptions
To from urllib3 import exceptions
Works now.