man-group / ArcticDB

ArcticDB is a high performance, serverless DataFrame database built for the Python Data Science ecosystem.
http://arcticdb.io
Other
1.23k stars 79 forks source link

Decommission Python 3.6 build #1510

Open poodlewars opened 2 months ago

poodlewars commented 2 months ago

No longer required.

Open question - what Pandas compat do we need? 3.6 build is the only one that tests Pandas 0.x compat. It would help the projects velocity to stop supporting Pandas 0.

This query in BigQuery shows that version 0 is extremely unpopular:

SELECT substring(file.version, 1, 1), COUNT(*) AS num_downloads
FROM `bigquery-public-data.pypi.file_downloads`
WHERE file.project = 'pandas'
  -- Only query the last 30 days of history
  AND DATE(timestamp)
    BETWEEN DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
    AND CURRENT_DATE()
group by substring(file.version, 1, 1)
order by 1
    ;

It hasn't been updated for 5 years.

Image

I think it's rational to remove support for it. Alternatively, we could test against it in the Python 3.8 build.

poodlewars commented 2 months ago

We need to be careful to continue to test backwards compat with Python 3.6 and Pandas 0. We should still have part of the CI that writes data in Python 3.6 with Pandas 0 and verify that later versions can work with it correctly. This is to make sure we don't break compat with already serialized data.

poodlewars commented 1 month ago

Additionally, some of the "analysis" workflow (eg ASV benchmarks) run with Python 3.6 at the moment. They will need to be upgraded to a modern Python.