search

scrapinghub / python-scrapinghub

A client interface for Scrapinghub's API
https://python-scrapinghub.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
202 stars 63 forks source link

Drop versions earlier than Python 3.7 and update requirements #159

Closed apalala closed 2 years ago

apalala commented 2 years ago

library upgrades

This update nominally drops support for Python 2.7, 3.5, and 3.6, and tests support for 3.10 to avoid libraries being pinned to very old versions, many of them with bugs or with security issues.

It's "nominally" because the code hasn't been changed except for deprecations enforced in Python 3.9 or 3.10.

disabled tests

Tests that required running test servers are disabled:

The tests can be re-enabled by someone with access to test servers.

jdemaeyer commented 2 years ago

shub no longer works on Arch Linux as Python 3.10 landed in the official repos a week ago (breaking wihle trying to run the no-longer-possible imports from collections instead of collections.abc). There has been no activity on this PR for two months, any chance it will be merged soon?

pawelmhm commented 2 years ago

we need to move this forward, tests are failing becuase of cassette replay problems. ~on storage.vm.scrapinghub. I'm not sure why we have to rely on making connections to some development server, this is a recipe for trouble, is there no way to fake tests here?~

I also don't think we need to bother about upgrading all libraries and dropping python 2 support NOW. We need to have this working with python 3.10 asap and worry about other things later. I'll try to make other smaller PR with only essential things

(edited)

apalala commented 2 years ago

@pawelmhm, the upgrade on this pull request is all we need. I know because I've tested it enough.

Everything else can wait.

elacuesta commented 2 years ago

Python 3.10 support added in #166