search

datamade / scrapers-us-municipal

Scrapers for US municipal governments.
MIT License
10 stars 8 forks source link

Can't `pip install -r requirements.txt --upgrade` because our python is too old #40

Closed fgregg closed 4 years ago

fgregg commented 4 years ago

Our server has python 3.4 installed.

When we run it fails on upgrading lxml

> pip install -r requirements.txt --upgrade
DEPRECATION: Python 3.4 support has been deprecated. pip 19.1 will be the last one supporting it. Please upgrade your Python as Python 3.4 won't be maintained after March 2019 (cf PEP 429).
Collecting https://github.com/opencivicdata/python-opencivicdata-django/archive/master.zip (from -r requirements.txt (line 1))
  Downloading https://github.com/opencivicdata/python-opencivicdata-django/archive/master.zip
     - 92kB 2.2MB/s
Collecting https://github.com/opencivicdata/python-legistar-scraper/archive/master.zip (from -r requirements.txt (line 3))
  Downloading https://github.com/opencivicdata/python-legistar-scraper/archive/master.zip (1.4MB)
     |████████████████████████████████| 1.4MB 15.1MB/s 
Collecting https://github.com/datamade/django-councilmatic/archive/master.zip (from -r requirements.txt (line 9))
  Downloading https://github.com/datamade/django-councilmatic/archive/master.zip
     \ 3.3MB 8.6MB/s
Requirement already up-to-date: pupa==0.9.0 in /home/datamade/.virtualenvs/opencivicdata/lib/python3.4/site-packages (from -r requirements.txt (line 2)) (0.9.0)
Collecting lxml (from -r requirements.txt (line 4))
  Using cached https://files.pythonhosted.org/packages/e1/f5/5eb3b491958dcfdcfa5daae3c655ab59276bc216ca015e44743c9c220e9e/lxml-4.4.0.tar.gz
    ERROR: Complete output from command python setup.py egg_info:
    ERROR: This lxml version requires Python 2.7, 3.5 or later.
    ----------------------------------------
ERROR: Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-2ytwk0s6/lxml/

This prevents anything else from getting upgraded, most critically, python-legistar-scraper.

We could address as follows:

  1. pin a version of lxml
  2. upgrade python3.5 (we can do this with just apt-get upgrade python3.5)
fgregg commented 4 years ago

okay, we have gotten into a critical dependency hell. I'm going to take the second path here.

https://sentry.io/organizations/datamade/issues/1294711177/?project=56420&referrer=slack

fgregg commented 4 years ago

we are now running on python3.5