Support CPython 3.10, drop CPython 3.5

[PMeira]

The final release is expected very soon (2021-10-04)

[PMeira]

Number of downloads for Python 3.5 is very low, should be safe to remove it soon. The majority of users seems to use 3.7 or 3.8.

[kdheepak]

Is there something I can do to help publish a release for support for 3.10?

[kdheepak]

I think it would be nice to not use the python version as part of the pypi wheel tag, e.g. replace cp39-cp39 with py3. Right now when a new version of Python is released, users get a very old version of opendssdirect. This is what happens when I remove the dependency dss_python from opendssdirect and upgrade my install:

Installing collected packages: OpenDSSDirect.py
  Attempting uninstall: OpenDSSDirect.py
    Found existing installation: OpenDSSDirect.py 0.2.14
    Uninstalling OpenDSSDirect.py-0.2.14:
      Successfully uninstalled OpenDSSDirect.py-0.2.14
  Running setup.py develop for OpenDSSDirect.py
Successfully installed OpenDSSDirect.py-0.6.1

[kdheepak]

[kdheepak]

For what it is worth, I downloaded the 3.9 wheel and renamed the file to dss_python-0.10.7.post1-py3-none-macosx_10_9_x86_64.whl, and it worked in 3.10.

[PMeira]

I guess just adding wheels for the released versions shouldn't present any issues. Just remember that we can't replace the wheels without incrementing the version tag.

For what it is worth, I downloaded the 3.9 wheel and renamed the file to dss_python-0.10.7.post1-py3-none-macosx_10_9_x86_64.whl, and it worked in 3.10.

We can't rely on that. The CPython limited API would allow that for CFFI and remove the requirement of building separate wheels -- https://github.com/dss-extensions/dss_python/issues/24 -- but Cython still doesn't support it.

We could consider separating the compiled parts of DSS Python into separate modules -- one for the DSS C-API libraries, one for the CFFI backend, one for the Cython backend (this would be good for conda-forge too) -- and use whatever backend is available during import time. Using the limited API, the CFFI backend would be future-proof for a while.

Yet another option is to add a proper source release for DSS Python that downloads the Pascal binaries and builds everything locally. I never did that because nobody asked so far.

I should finally have more time starting next week (big project ending). I'll send you an email regarding lots of changes that need be integrated soon for the next DSS C-API (a lot besides what's in the master branch there). Probably doesn't make sense to launch the new version during the holiday season, but an alpha or beta would be useful to start testing things.

[kdheepak]

I guess just adding wheels for the released versions shouldn't present any issues.

Great! This works for me. I wonder if there’s some automated way of doing it for releases.

I think a source release would be good but I’m hesitant because people will have a frustrating experience if things don’t work. But also, right now pip silently downloading a older version of OpenDSSDirect is frustrating too. I wish pip / conda said something like “latest version is XXX, but installing YYY because of these constraints”.

I’m also making some helper classes for OpenDSSDirect that should make it easier to get data in a format that user expect. I’ll make a PR and tag you for review.

[beroset]

I'm also interested in this. I'm attempting to build dss_python with the latest github version of dss_capi. I'd be happy to contribute a PR for that if there's any interest. Is there?

[PMeira]

For what it is worth, I downloaded the 3.9 wheel and renamed the file to dss_python-0.10.7.post1-py3-none-macosx_10_9_x86_64.whl, and it worked in 3.10.

@kdheepak Good news: the extension modules were all built with Py_LIMITED_API, including the Windows ones. CFFI turned that on a couple of releases ago.

I downloaded the files (Py 3.9, 64-bit for Windows, Linux and macOS), edited the metadata (at least the WHEEL file needs to be updated, but updating METADATA and RECORD is good too), renamed the .whls, uploaded them to TestPyPI, and tested them just to be sure. Since everything seems to work, I uploaded to the main PyPI.

I wonder if there’s some automated way of doing it for releases.

I didn't find any specific tool for that, but certainly looks like it could be automated without too much trouble.

I'm attempting to build dss_python with the latest github version of dss_capi. I'd be happy to contribute a PR for that if there's any interest. Is there?

@beroset I can merge some of my local changes for DSS Python and prepare a preview release for the latest public version of DSS C-API. I'll keep working a bit on finishing my local work on DSS C-API during the holidays. A lot of code has been rewritten, so I might just squash nearly everything before pushing the code and preparing another pre-release for more extensive testing.

[beroset]

Sounds good to me. Let me know if I can be of any help with this.

[PMeira]

Happy New Year!

I just uploaded that first pre-release at https://pypi.org/project/dss-python/0.12.0a1/#files These use Py_LIMITED_API as before but with the correct options during setup ( --py-limited-api cp36), so a single wheel per platform should work fine from Python 3.6 up to 3.10. For ARM (32 and 64-bit), for the time being, I'm building locally, everything else was migrated to GitHub Actions.

If Cython doesn't work with Py_LIMITED_API yet, it could be worth to create a separate package just for the future Cython backend and keep it optional.

For Anaconda, it seems Python 3.10 support is not there yet, but conda-forge seems OK. conda-build (and boa) is slow (>20 min) and error-prone, so I left the builds disabled. It might be worth adding Actions workflows to handle each platform separately, but I'm fine with building the packages locally when a stable release is ready.