search

sfu-db / connector-x

Fastest library to load data from DB to DataFrames in Rust and Python
https://sfu-db.github.io/connector-x
MIT License
1.85k stars 146 forks source link

Please don't delete releases from PyPI #630

Closed rafmagns-skepa-dreag closed 1 month ago

rafmagns-skepa-dreag commented 1 month ago

It looks like as part of the release of 0.3.3 old alpha versions of connectorx were deleted from pypi. I was still using 0.3.2a2 because of the reasons listed in #539 and this broke my builds and deployments. by sheer luck I'm also in the process of upgrading my boxes to something that has a much newer version of libc. If you must remove things from pypi, please yank them - don't delete them. And also, please don't do that! I noticed that there are also no releases or tags on github for those 0.3.2 alpha releases so I couldn't reproduce exactly what I had before even if I tried

AndrewJackson2020 commented 1 month ago

I agree, Love to see new releases but deleting old releases severely inhibits reproducibility. Old releases should only be deleted under extreme conditions like a security vulnerability or the like and should be publicized.

wangxiaoying commented 1 month ago

Indeed. The old alpha versions are deleted since we reached to the total project size limit of 10GB in PyPI and new versions cannot be uploaded to it. The rewriter jar is the main reason that the current wheel file is 50-60MB each (where we maintain 4 python versions on 4 platforms, so each version on PyPI is ~900MB). We probably need to move it to an independent project instead of embed it into connectorx.

rafmagns-skepa-dreag commented 1 month ago

oh wow I didn't realize! thanks for the explanation

evbo commented 1 month ago

@rafmagns-skepa-dreag any workaround to this? I'm also on Centos 7 still (ugh...) and am dead in the water without 0.3.2a2

AndrewJackson2020 commented 1 month ago

@rafmagns-skepa-dreag any workaround to this? I'm also on Centos 7 still (ugh...) and am dead in the water without 0.3.2a2

We found a previously built artifact that we are using but that doesn't help you.

@wangxiaoying Any chance that you can restore the tag in github that was used to build 0.3.2a2 so that people can build their own release of the alpha build even if you are unable to host it on PyPI?

evbo commented 1 month ago

I'm able to avoid connectorx by using pymysql to get results as a list of dicts that polars can read.

Ugly, slow, but works until I can get off centos 7...