deepgenomics / GenomeKit

A Python library for fast and easy access to genomic resources such as sequence, data tracks, and annotations
Apache License 2.0
33 stars 2 forks source link

support python 311 312 313 #102

Open ovesh opened 1 month ago

ovesh commented 1 month ago

Requires updating meta.yaml (on conda-forge, but only after local testing)

ovesh commented 1 month ago

Started the work (including deprecation of py38 which lost support on conda-forge) but ran into multiple issues.

When py312 is included, conda mambabuild --croot ~/conda-bld -q --no-test conda-recipe eventually errors out with nothing provides _python_rc needed by python-3.12.0rc3-rc3_hab00c5b_1_cpython.

When just py311 is added, conda mambabuild --croot /tmp/conda-bld -t ~/conda-bld/*/*py311*.tar.bz2 --extra-deps python=3.11 (running the tests) fails with ModuleNotFoundError: No module named 'numpy'.

When creating a local py312 env, you get this error on importing gk: SystemError: type genome_kit._cxx.GenomeTrackBuilder has the Py_TPFLAGS_HAVE_GC flag but has no traverse function. Caused by this change in py311:

The PyType_Ready() function now raises an error if a type is defined with the Py_TPFLAGS_HAVE_GC flag set but has no traverse function (PyTypeObject.tp_traverse). (Contributed by Victor Stinner in bpo-44263.)

ovesh commented 3 weeks ago

Got a segfault on exiting python in py312 and py313:

# python
Python 3.12.7 | packaged by conda-forge | (main, Oct  4 2024, 15:55:29) [Clang 17.0.6 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import genome_kit
>>> genome_kit.Genome('hg19').interval('chr1','+',0,1)
Interval("chr1", "+", 0, 1, "hg19")
>>>
zsh: segmentation fault  python

It happens when this static unordered map is deallocated, specifically when Py_XDECREF is called on the stored PyObject str.

Looked into it and it's likely a python bug, for now I created https://github.com/python/cpython/issues/126508 If we can't get this bug fixed we'll need to work around it somehow.