Closed tbaumann closed 1 year ago
I have not seen that behaviour before.
Just to discard an issue with the current version. Could you try and older release juju==2.9.10
or juju==3.0.4
to check if it keeps failing?
Interesting.
juju==2.9.10
WARNING WARNING:juju.client.connection:unknown facade EnvironUpgrader connection.py:873
WARNING WARNING:juju.client.connection:unexpected facade EnvironUpgrader found, unable to decipher version to use connection.py:897
DEBUG DEBUG:juju.client.connection:connection 140639884821216 -> { connection.py:597
"type": "Pinger",
"request": "Ping",
"version": 1,
"params": {},
"request-id": 8
}
DEBUG DEBUG:websockets.protocol:client > Frame(fin=True, opcode=1, data=b'{\n "type": "Pinger",\n "request": "Ping",\n "version": 1,\n "params": protocol.py:898
{},\n "request-id": 8\n}', rsv1=False, rsv2=False, rsv3=False)
DEBUG DEBUG:websockets.protocol:client > Frame(fin=True, opcode=1, data=b'{\n "type": "Pinger",\n "request": "Ping",\n "version": 1,\n "params": protocol.py:898
{},\n "request-id": 6\n}', rsv1=False, rsv2=False, rsv3=False)
DEBUG DEBUG:websockets.protocol:client < Frame(fin=True, opcode=1, data=b'{"request-id":8,"response":{}}\n', rsv1=False, rsv2=False, rsv3=False) protocol.py:886
DEBUG DEBUG:websockets.protocol:client < Frame(fin=True, opcode=1, data=b'{"request-id":6,"response":{}}\n', rsv1=False, rsv2=False, rsv3=False) protocol.py:886
DEBUG DEBUG:juju.client.connection:connection 140639884821216 <- {'request-id': 8, 'response': {}} connection.py:620
DEBUG DEBUG:juju.client.connection:connection 140639884821216 <- {'request-id': 6, 'response': {}} connection.py:620
⠼ Connecting to model[02/27/23 13:28:12] DEBUG DEBUG:juju.client.connection:connection 140639884821216 -> { connection.py:597
"type": "Pinger",
"request": "Ping",
"version": 1,
"params": {},
"request-id": 9
}
DEBUG DEBUG:websockets.protocol:client > Frame(fin=True, opcode=1, data=b'{\n "type": "Pinger",\n "request": "Ping",\n "version": 1,\n "params": protocol.py:898
{},\n "request-id": 9\n}', rsv1=False, rsv2=False, rsv3=False)
DEBUG DEBUG:websockets.protocol:client < Frame(fin=True, opcode=1, data=b'{"request-id":9,"response":{}}\n', rsv1=False, rsv2=False, rsv3=False) protocol.py:886
DEBUG DEBUG:juju.client.connection:connection 140639884821216 <- {'request-id': 9, 'response': {}} connection.py:620
⠧ Connecting to model[02/27/23 13:28:22] DEBUG DEBUG:websockets.protocol:client > Frame(fin=True, opcode=9, data=b'\x90s.5', rsv1=False, rsv2=False, rsv3=False) protocol.py:898
DEBUG DEBUG:websockets.protocol:client < Frame(fin=True, opcode=10, data=b'\x90s.5', rsv1=False, rsv2=False, rsv3=False) protocol.py:886
DEBUG DEBUG:websockets.protocol:client - received solicited pong: 90732e35 protocol.py:848
⠏ Connecting to model DEBUG DEBUG:juju.client.connection:connection 140639884821216 -> { connection.py:597
"type": "Pinger",
"request": "Ping",
"version": 1,
"params": {},
"request-id": 10
}
DEBUG DEBUG:websockets.protocol:client > Frame(fin=True, opcode=1, data=b'{\n "type": "Pinger",\n "request": "Ping",\n "version": 1,\n "params": protocol.py:898
{},\n "request-id": 10\n}', rsv1=False, rsv2=False, rsv3=False)
DEBUG DEBUG:websockets.protocol:client < Frame(fin=True, opcode=1, data=b'{"request-id":10,"response":{}}\n', rsv1=False, rsv2=False, rsv3=False) protocol.py:886
DEBUG DEBUG:juju.client.connection:connection 140639884821216 <- {'request-id': 10, 'response': {}} connection.py:620
⠼ Connecting to model[02/27/23 13:28:32] DEBUG DEBUG:juju.client.connection:connection 140639884821216 -> { connection.py:597
"type": "Pinger",
"request": "Ping",
"version": 1,
"params": {},
"request-id": 11
}
DEBUG DEBUG:websockets.protocol:client > Frame(fin=True, opcode=1, data=b'{\n "type": "Pinger",\n "request": "Ping",\n "version": 1,\n "params": protocol.py:898
{},\n "request-id": 11\n}', rsv1=False, rsv2=False, rsv3=False)
DEBUG DEBUG:websockets.protocol:client < Frame(fin=True, opcode=1, data=b'{"request-id":11,"response":{}}\n', rsv1=False, rsv2=False, rsv3=False) protocol.py:886
DEBUG DEBUG:juju.client.connection:connection 140639884821216 <- {'request-id': 11, 'response': {}} connection.py:620
⠧ Connecting to model[02/27/23 13:28:42] DEBUG DEBUG:websockets.protocol:client > Frame(fin=True, opcode=9, data=b'\x11\xf5\x9cO', rsv1=False, rsv2=False, rsv3=False) protocol.py:898
⠇ Connecting to model DEBUG DEBUG:websockets.protocol:client < Frame(fin=True, opcode=10, data=b'\x11\xf5\x9cO', rsv1=False, rsv2=False, rsv3=False) protocol.py:886
DEBUG DEBUG:websockets.protocol:client - received solicited pong: 11f59c4f protocol.py:848
⠏ Connecting to model DEBUG DEBUG:juju.client.connection:connection 140639884821216 -> { connection.py:597
"type": "Pinger",
"request": "Ping",
"version": 1,
"params": {},
"request-id": 12
}
DEBUG DEBUG:websockets.protocol:client > Frame(fin=True, opcode=1, data=b'{\n "type": "Pinger",\n "request": "Ping",\n "version": 1,\n "params": protocol.py:898
{},\n "request-id": 12\n}', rsv1=False, rsv2=False, rsv3=False)
DEBUG DEBUG:websockets.protocol:client < Frame(fin=True, opcode=1, data=b'{"request-id":12,"response":{}}\n', rsv1=False, rsv2=False, rsv3=False) protocol.py:886
DEBUG DEBUG:juju.client.connection:connection 140639884821216 <- {'request-id': 12, 'response': {}} connection.py:620
So no change there.
But juju==3.0.4 works! This is really fascinating and unexpected since libjuju 3.X dropped support for 2.9.X juju... What should I use now as juju 2.9 user?
3.0.4 is a transition version, it still provides support for 2.9.
This is weird because 2.9.38.1 was tested with juju 2.9.38 and we didn't find this problem. I'm curious if this only happens for versions older than 2.9.38.
Which version should I use to retain support of juju 2.9? As I will be on that version for quite some time. I assumed whatever the latest libjuju 2.9.x is the latest is right.
So python-libjuju 3.0.* should support both 2.9 and 3.0, though it uses 3.0 semantics. However, that moves you closer to being ready to move to 3.1 in the future (3.1 is the one that actually drops support for 2.9).
Ah good, so I should target version 3.0 in my code. Because it has some API breaks vs. 2.9 which I had targeted with my code so far. I will do anything that will give me some level of stability with regards to compatibility.
Anyway, I've just tried with a fresh 2.9.33 controller and I could connect, add a model, and deploy an ubuntu charm. Couldn't reproduce this connectivity issue.
I will close this issue as there is a workaround for this and we can't reproduce it right now.
Unfortunately I'm not even surprised. Because this worked with my test models. Just the first time I ran it in 'production' it failed like this. But what could be the issue?
Anyway, if I can stay on version 3.0 then I probably don't need to know...
From the trace it looks like the server simply cut the connection. And this was done after the negotiation of the facades. I cannot say right now what could be the reason for this. It would probably require some additional investigation checking the controller logs.
Description
I just ran my code for the first time in production. The juju version is a little different but I expected to be fine with the latest 2.9 release.
First I tried
juju==2.9.11
and then laterjuju==2.9.38.1
.Connecting seems to hang in a loop. When I enable debug I see it reconnects to the API socket all the time.
Juju cli works just fine. Any idea what's causing this?
Urgency
Casually reporting
Python-libjuju version
2.9.38.1
Juju version
2.9.33
Reproduce / Test