search

crate-py / rpds

Python bindings to the Rust rpds crate for persistent data structures
https://rpds.readthedocs.io/
MIT License
35 stars 12 forks source link

PyO3 modules do not yet support subinterpreters #63

Closed pipiche38 closed 5 months ago

pipiche38 commented 5 months ago

I am using a python library (zigpy) which then is called in an embedded python framework. This is call several time and we are having this issue.

This isse is happening with some other python modules for which we have been able to downgrade to a level of PyO3 where the problem doesn't exist.


Apr 04 12:17:56 CasaiaProV4-test domoticz[19084]: Zigbee(reseau 2): Traceback (most recent call last):
Apr 04 12:17:56 CasaiaProV4-test domoticz[19084]: Zigbee(reseau 2):   File "/home/pi/domoticz/plugins/Domoticz-Zigbee/plugin.py", line 1540, in onStart
Apr 04 12:17:56 CasaiaProV4-test domoticz[19084]: Zigbee(reseau 2):     _plugin.onStart()
Apr 04 12:17:56 CasaiaProV4-test domoticz[19084]: Zigbee(reseau 2):   File "/home/pi/domoticz/plugins/Domoticz-Zigbee/plugin.py", line 555, in onStart
Apr 04 12:17:56 CasaiaProV4-test domoticz[19084]: Zigbee(reseau 2):     start_zigbee_transport(self )
Apr 04 12:17:56 CasaiaProV4-test domoticz[19084]: Zigbee(reseau 2):   File "/home/pi/domoticz/plugins/Domoticz-Zigbee/plugin.py", line 979, in start_zigbee_transport
Apr 04 12:17:56 CasaiaProV4-test domoticz[19084]: Zigbee(reseau 2):     _start_zigpy_ZNP(self)
Apr 04 12:17:56 CasaiaProV4-test domoticz[19084]: Zigbee(reseau 2):   File "/home/pi/domoticz/plugins/Domoticz-Zigbee/plugin.py", line 1069, in _start_zigpy_ZNP
Apr 04 12:17:56 CasaiaProV4-test domoticz[19084]: Zigbee(reseau 2):     from Classes.ZigpyTransport.Transport import ZigpyTransport
Apr 04 12:17:56 CasaiaProV4-test domoticz[19084]: Zigbee(reseau 2):   File "/home/pi/domoticz/plugins/Domoticz-Zigbee/Classes/ZigpyTransport/Transport.py", line 9, in <module>
Apr 04 12:17:56 CasaiaProV4-test domoticz[19084]: Zigbee(reseau 2):     import zigpy.application
Apr 04 12:17:56 CasaiaProV4-test domoticz[19084]: Zigbee(reseau 2):   File "/usr/local/lib/python3.9/dist-packages/zigpy/application.py", line 22, in <module>
Apr 04 12:17:56 CasaiaProV4-test domoticz[19084]: Zigbee(reseau 2):     import zigpy.appdb
Apr 04 12:17:56 CasaiaProV4-test domoticz[19084]: Zigbee(reseau 2):   File "/usr/local/lib/python3.9/dist-packages/zigpy/appdb.py", line 16, in <module>
Apr 04 12:17:56 CasaiaProV4-test domoticz[19084]: Zigbee(reseau 2):     import zigpy.device
Apr 04 12:17:56 CasaiaProV4-test domoticz[19084]: Zigbee(reseau 2):   File "/usr/local/lib/python3.9/dist-packages/zigpy/device.py", line 12, in <module>
Apr 04 12:17:56 CasaiaProV4-test domoticz[19084]: Zigbee(reseau 2):     from zigpy.ota.manager import find_ota_cluster, update_firmware
Apr 04 12:17:56 CasaiaProV4-test domoticz[19084]: Zigbee(reseau 2):   File "/usr/local/lib/python3.9/dist-packages/zigpy/ota/__init__.py", line 27, in <module>
Apr 04 12:17:56 CasaiaProV4-test domoticz[19084]: Zigbee(reseau 2):     import zigpy.ota.providers
Apr 04 12:17:56 CasaiaProV4-test domoticz[19084]: Zigbee(reseau 2):   File "/usr/local/lib/python3.9/dist-packages/zigpy/ota/providers.py", line 19, in <module>
Apr 04 12:17:56 CasaiaProV4-test domoticz[19084]: Zigbee(reseau 2):     import jsonschema
Apr 04 12:17:56 CasaiaProV4-test domoticz[19084]: Zigbee(reseau 2):   File "/usr/local/lib/python3.9/dist-packages/jsonschema/__init__.py", line 13, in <module>
Apr 04 12:17:56 CasaiaProV4-test domoticz[19084]: Zigbee(reseau 2):     from jsonschema._format import FormatChecker
Apr 04 12:17:56 CasaiaProV4-test domoticz[19084]: Zigbee(reseau 2):   File "/usr/local/lib/python3.9/dist-packages/jsonschema/_format.py", line 11, in <module>
Apr 04 12:17:56 CasaiaProV4-test domoticz[19084]: Zigbee(reseau 2):     from jsonschema.exceptions import FormatError
Apr 04 12:17:56 CasaiaProV4-test domoticz[19084]: Zigbee(reseau 2):   File "/usr/local/lib/python3.9/dist-packages/jsonschema/exceptions.py", line 15, in <module>
Apr 04 12:17:56 CasaiaProV4-test domoticz[19084]: Zigbee(reseau 2):     from referencing.exceptions import Unresolvable as _Unresolvable
Apr 04 12:17:56 CasaiaProV4-test domoticz[19084]: Zigbee(reseau 2):   File "/usr/local/lib/python3.9/dist-packages/referencing/__init__.py", line 4, in <module>
Apr 04 12:17:56 CasaiaProV4-test domoticz[19084]: Zigbee(reseau 2):     from referencing._core import Anchor, Registry, Resource, Specification
Apr 04 12:17:56 CasaiaProV4-test domoticz[19084]: Zigbee(reseau 2):   File "/usr/local/lib/python3.9/dist-packages/referencing/_core.py", line 8, in <module>
Apr 04 12:17:56 CasaiaProV4-test domoticz[19084]: Zigbee(reseau 2):     from rpds import HashTrieMap, HashTrieSet, List
Apr 04 12:17:56 CasaiaProV4-test domoticz[19084]: Zigbee(reseau 2):   File "/usr/local/lib/python3.9/dist-packages/rpds/__init__.py", line 1, in <module>
Apr 04 12:17:56 CasaiaProV4-test domoticz[19084]: Zigbee(reseau 2):     from .rpds import *
Apr 04 12:17:56 CasaiaProV4-test domoticz[19084]: Zigbee(reseau 2): ImportError: PyO3 modules do not yet support subinterpreters, see https://github.com/PyO3/pyo3/issues/576
pipiche38 commented 5 months ago

https://github.com/PyO3/pyo3/issues/576

is this approach possible https://github.com/pyca/bcrypt/pull/714 ?

Julian commented 5 months ago

Hi. Sorry you're having trouble.

I'm not sure what the intention of this issue is though.

That error message happens if you initialize a module multiple times, and is saying "that's not supported (yet)". In a "normal" Python situation it shouldn't happen, so something in your setup is doing it. I don't know whether or what you should change, and it's possible you might indeed need to wait for PyO3 to eventually support this, or it's possible you're doing something incorrectly in initialization, but it's hard to guess.