search

jkeljo / sisyphus-control

Control your Sisyphus Kinetic Art Table from Python 3.6 with asyncio
Other
10 stars 2 forks source link

Compatibility with new `python-socketio` and `python-engineio` versions? #6

Closed cdce8p closed 2 months ago

cdce8p commented 2 years ago

I'm current working on resolving potential dependency conflicts in Home Assistant. While doing that, I noticed that the latest versions for both python-socketio and python-engineio aren't supported. For the moment, there seems to be a solution but in the long term this might conflict with aioambient which also uses socketio. Would you be able to take a look if you can bump the dependency range?

https://github.com/jkeljo/sisyphus-control/blob/5f53068c6cbbade6bf20462710461128fa633684/setup.py#L31-L36

The current non-conflicting versions constraints (in Home Assistant).

python-engineio>=3.13.1,<4.0
python-socketio>=4.6.0,<5.0

The current constraints from aioambient: https://github.com/bachya/aioambient/blob/2021.11.0/pyproject.toml#L63-L64

Changelog python-socketio: https://github.com/miguelgrinberg/python-socketio/blob/main/CHANGES.md Changelog python-engineio: https://github.com/miguelgrinberg/python-engineio/blob/main/CHANGES.md

jkeljo commented 2 years ago

Unfortunately I can't. :( The maintainers of engineio/socketio decided to tie their version numbers to the protocol versions, which are not backward compatible. Sisyphus tables use the 2.0 protocol, so that locks me into older versions of the libraries.

A few options I can think of:

jkeljo commented 2 years ago

Or we could ask the socketio/engineio maintainers to publish the older versions under different names.

cdce8p commented 2 years ago

Unfortunately I can't. :( The maintainers of engineio/socketio decided to tie their version numbers to the protocol versions, which are not backward compatible. Sisyphus tables use the 2.0 protocol, so that locks me into older versions of the libraries.

That is truly unfortunate 😞

  • ask the owners of engineio/socketio to support older protocols in the latest libraries, so that single apps like Home Assistant that need to communicate using multiple versions of the protocol can do so
  • Or we could ask the socketio/engineio maintainers to publish the older versions under different names.

Don't know that would result in anything. It seems to be an active choice to do it that way. If they wanted to do it differently, I would think that they would have done so already.

  • come up with some Python loading magic that would let different HA integrations use different dependency versions

Some kind of dependency isolation? That would certainly help, but I'm not aware of an easy solution within the current Python ecosystem. So I think that's off the table.

  • make a Home Assistant plug-in that connects the older socketio protocols to mqtt or something, and have integrations that need the older protocol use that

Pretty cumbersome for something that already works with an older version.

  • fork the older versions into libraries with different names

IMO this would probably be the solution here. In addition to the library name, the main package name would need to change to but that's strait forward. Long term this might be the best way forward as it would allow packages that depend on both protocol versions to coexist. Maybe you would be interested in pursuing this path.

I won't be able to help with with that unfortunately.

Just for socketio the tasks would probably be something like:

The name was just my first idea. I assume you probably know better which one would fit.

-- For now the PR I opened should work. So there isn't a direct need. If at some point a dependency creates an actually conflict though, the only remaining option would probably be to move one integration from core to HACS with the explicit mention that they can't be used together.

onkelbeh commented 2 years ago

If you are going to use it with Home Assistant, as you say, this problem won't show up anyway, because HA currently (2022.2.0b3) limits

python-engineio>=3.13.1,<4.0
python-socketio>=4.6.0,<5.0

itself.

cdce8p commented 2 years ago

If you are going to use it with Home Assistant, as you say, this problem won't show up anyway, because HA currently (2022.2.0b3) limits

python-engineio>=3.13.1,<4.0
python-socketio>=4.6.0,<5.0

itself.

I added the constraints myself as response to this issue. For now it works. In the long run, we might want to use packages that require python-engineio>=4.0 / python-socketio>=5.0 which isn't possible currently. That's why ideally I would like to remove the constraints at some point.

schnabel commented 2 years ago

Hi @jkeljo and @cdce8p as you might have seen, I ran into the expected problems with the pinned (old) versions of socketio and engineio with pyhifiberry. I had a discussion with the author of socketio where he recommended to copy the version of socketio and engineio into your own application/library. This is what I did with pyhifiberry and it works without any problems. I think it would be a good decision though, that this patching is done in the integrations/libraries that use the old versions of socketio. That way the currently supported version is not blocked for everybody else. What do you think? I could offer a PR, but unfortunately I do not have a sisyphus table :-( so I cannot test it. The patching was done in a few minutes and as mentioned above is just a matter of copying the code and changing some imports as you can see in pyhifiberry.

jkeljo commented 2 years ago

I think it would be a good decision though, that this patching is done in the integrations/libraries that use the old versions of socketio.

I think we should probably just do it in all of them. Today's current versions are tomorrows old versions, after all, and it doesn't sound like they're changing their versioning practices. I'll definitely look into doing that here.

schnabel commented 2 years ago

Thanks for willing to take a look at it :-). I am not sure if I agree to do it in all integrations. I think the versioning practice of python-socketio is just fine and not special. Breaking API changes do occur all the time and they are usually reflected in the major version number. The problem we are facing may happen in integration platform like HA a lot and as mentioned above is actually a problem of dependency isolation. IMO it is a good practice to try to stay up to date and hope for stable APIs.

jkeljo commented 2 years ago

I agree with you for normal API breaking changes where you can just update your code to use the new APIs. The problem with socketio's versioning is that it ties the major version to a version of a wire protocol, so you can't just update your client until all servers you might be working with have upgraded (and at the same time, the servers can't update until your client has). If you control both client and server that's fine, but for HA integrations we usually don't. Your library is on the latest version now, but if a new version of the socketio protocol comes out and hifiberry doesn't switch to it, you'll be in the same spot as I am today. (And if hifiberry does switch to it, you'll be in the tough spot of having to either force people to upgrade to continue using your integration, or somehow manage backward compatibility yourself since pysocketio has declined to do so.)

As an analogy, if you applied the py-socketio versioning scheme to something like OpenSSL, you would need to use different major versions of OpenSSL for SSL 1.0, 2.0, 3.0, and TLS 1.x. Of course that's not how it works; one version of OpenSSL supports everything from SSL2 forward (and SSL1 wasn't removed at the exact time that SSL2 was added).

schnabel commented 2 years ago

Still not really convinced. But I am fine with different opinions :-). I think it is good to try to be backward compatible if feasible. But I do not think it is wise to have that expectation on remote APIs in the IOT world.

jsv0ice commented 10 months ago

I'd like to bump this issue, I'm developing a home assistant component and am running into this constraint.

bdraco commented 2 months ago

https://pypi.org/project/python-socketio-v4/ has been published and can be used as an alternative