Realtime API closes immediately

[bachya]

I'm attempting to connect to the Realtime API in a Python package (using python-socketio). When I attempt a simple connection:

sio = socketio.AsyncClient()
await sio.connect('https://api.ambientweather.net/?api=1&applicationKey=<MY_APP_KEY>')

...I get the follow stacktrace:

INFO:engineio.client:Attempting polling connection to https://api.ambientweather.net/socket.io/?api=1&applicationKey=<MY_APP_KEY>&transport=polling&EIO=3
INFO:engineio.client:Polling connection accepted with {'sid': '<GENERATED_SID>', 'upgrades': ['websocket'], 'pingInterval': 25000, 'pingTimeout': 5000}
INFO:socketio.client:Engine.IO connection established
INFO:engineio.client:Attempting WebSocket upgrade to wss://api.ambientweather.net/socket.io/?api=1&applicationKey=<MY_APP_KEY>&transport=websocket&EIO=3
DEBUG:websockets.protocol:client - state = CONNECTING
DEBUG:websockets.protocol:client - event = connection_made(<asyncio.sslproto._SSLProtocolTransport object at 0x109ef8128>)
DEBUG:websockets.protocol:client ! failing WebSocket connection in the CONNECTING state: 1006 [no reason]
DEBUG:websockets.protocol:client x closing TCP connection
DEBUG:websockets.protocol:client - event = connection_lost(None)
DEBUG:websockets.protocol:client - state = CLOSED
DEBUG:websockets.protocol:client x code = 1006, reason = [no reason]
WARNING:engineio.client:WebSocket upgrade failed: connection error
INFO:engineio.client:Sending packet PING data None
INFO:engineio.client:Sending polling GET request to https://api.ambientweather.net/socket.io/?api=1&applicationKey=<MY_APP_KEY>&transport=polling&EIO=3&sid=<GENERATED_SID>

The CONNECTING failure happens fairly immediately.

Not asking anyone to debug the Python library, per se, but I'm at a loss as to how I debug this further. 1006 isn't giving me much to work off of. Any thoughts on where I can go?

[bachya]

I saw from this comment that perhaps dash2.ambientweather.net was a better URL to use, so I went that route. That URL seems to work better, but it's still sporadic – 2 out of every 5 tries, I get a 400 response; the other 3 tries, I get data back, but after about 20-30 seconds, the connection closes.

Example new stacktrace:

INFO:engineio.client:Attempting polling connection to https://dash2.ambientweather.net/socket.io/?api=1&applicationKey=<MY_APP_KEY>&transport=polling&EIO=3
INFO:engineio.client:Polling connection accepted with {'sid': '<GENERATED_SID>', 'upgrades': ['websocket'], 'pingInterval': 25000, 'pingTimeout': 5000}
INFO:socketio.client:Engine.IO connection established
INFO:engineio.client:Attempting WebSocket upgrade to wss://dash2.ambientweather.net/socket.io/?api=1&applicationKey=<MY_APP_KEY>&transport=websocket&EIO=3
DEBUG:websockets.protocol:client - state = CONNECTING
DEBUG:websockets.protocol:client - event = connection_made(<asyncio.sslproto._SSLProtocolTransport object at 0x109739390>)
DEBUG:websockets.protocol:client - state = OPEN
DEBUG:websockets.protocol:client > Frame(fin=True, opcode=1, data=b'2probe', rsv1=False, rsv2=False, rsv3=False)
DEBUG:websockets.protocol:client < Frame(fin=True, opcode=1, data=b'3probe', rsv1=False, rsv2=False, rsv3=False)
DEBUG:websockets.protocol:client > Frame(fin=True, opcode=1, data=b'5', rsv1=False, rsv2=False, rsv3=False)
INFO:engineio.client:WebSocket upgrade was successful
INFO:socketio.client:Emitting event "subscribe" [/]
INFO:engineio.client:Sending packet MESSAGE data 2["subscribe",{"apiKeys":["<MY_API_KEY>"]}]
INFO:engineio.client:Sending packet PING data None
DEBUG:websockets.protocol:client > Frame(fin=True, opcode=1, data=b'42["subscribe",{"apiKeys":["<MY_API_KEY>"]}]', rsv1=False, rsv2=False, rsv3=False)
DEBUG:websockets.protocol:client > Frame(fin=True, opcode=1, data=b'2', rsv1=False, rsv2=False, rsv3=False)
DEBUG:websockets.protocol:client < Frame(fin=True, opcode=1, data=b'40', rsv1=False, rsv2=False, rsv3=False)
INFO:engineio.client:Received packet MESSAGE data 0
INFO:socketio.client:Namespace / is connected
DEBUG:websockets.protocol:client < Frame(fin=True, opcode=1, data=b'3', rsv1=False, rsv2=False, rsv3=False)
INFO:engineio.client:Received packet PONG data None
DEBUG:websockets.protocol:client < Frame(fin=True, opcode=1, data=b'42["subscribed",{"devices":[{"macAddress":"84:F3:EB:21:90:C4","lastData":{"dateutc":1546998300000,"winddir":341,"windspeedmph":0,"windgustmph":0,"maxdailygust":9.2,"tempf":36,"hourlyrainin":0,"eventrainin":0,"dailyrainin":0,"weeklyrainin":0,"monthlyrainin":0,"totalrainin":0,"baromrelin":30.6,"baromabsin":25.11,"humidity":41,"tempinf":70.9,"humidityin":29,"uv":0,"solarradiation":0,"feelsLike":36,"dewPoint":14.61,"deviceId":"5c328e3bc4eac477f09b7dc1","date":"2019-01-09T01:45:00.000Z"},"info":{"name":"Home","location":"Home"},"apiKey":"<MY_API_KEY>"}],"method":"subscribe"}]', rsv1=False, rsv2=False, rsv3=False)
INFO:engineio.client:Received packet MESSAGE data 2["subscribed",{"devices":[{"macAddress":"84:F3:EB:21:90:C4","lastData":{"dateutc":1546998300000,"winddir":341,"windspeedmph":0,"windgustmph":0,"maxdailygust":9.2,"tempf":36,"hourlyrainin":0,"eventrainin":0,"dailyrainin":0,"weeklyrainin":0,"monthlyrainin":0,"totalrainin":0,"baromrelin":30.6,"baromabsin":25.11,"humidity":41,"tempinf":70.9,"humidityin":29,"uv":0,"solarradiation":0,"feelsLike":36,"dewPoint":14.61,"deviceId":"5c328e3bc4eac477f09b7dc1","date":"2019-01-09T01:45:00.000Z"},"info":{"name":"Home","location":"Home"},"apiKey":"<MY_API_KEY>"}],"method":"subscribe"}]
INFO:socketio.client:Received event "subscribed" [/]
DEBUG:websockets.protocol:client > Frame(fin=True, opcode=9, data=b'\xae\x81=|', rsv1=False, rsv2=False, rsv3=False)
DEBUG:websockets.protocol:client < Frame(fin=True, opcode=10, data=b'\xae\x81=|', rsv1=False, rsv2=False, rsv3=False)
DEBUG:websockets.protocol:client - received solicited pong: ae813d7c
INFO:engineio.client:Sending packet PING data None
ERROR:engineio.client:packet queue is empty, aborting
INFO:engineio.client:Exiting write loop task
DEBUG:websockets.protocol:client < Frame(fin=True, opcode=8, data=b'', rsv1=False, rsv2=False, rsv3=False)
DEBUG:websockets.protocol:client - state = CLOSING
DEBUG:websockets.protocol:client > Frame(fin=True, opcode=8, data=b'', rsv1=False, rsv2=False, rsv3=False)
WARNING:engineio.client:Read loop: WebSocket connection was closed, aborting
INFO:engineio.client:Waiting for write loop task to end
INFO:engineio.client:Waiting for ping loop task to end
INFO:engineio.client:Exiting ping task
INFO:engineio.client:Exiting read loop task
DEBUG:websockets.protocol:client - event = eof_received()
DEBUG:websockets.protocol:client - event = connection_lost(None)
DEBUG:websockets.protocol:client - state = CLOSED
DEBUG:websockets.protocol:client x code = 1005, reason = [no reason]
DEBUG:websockets.protocol:client x closing TCP connection

Notice the packet queue is empty, aborting message near the end of the trace.

[owise1]

We'll need to look into this. And yes, for now using dash2.ambientweather.net should give you better results.

[bachya]

Thanks, @owise1. If I can provide anything that would help identify my connections and traffic in your logs, please let me know.

[bachya]

@owise1 Any news?

[owise1]

I ran the nodejs implementation of the realtime api for an hour and a half (using api.ambientweather.net) and didn't have any disconnects. The server side uses the nodejs socket.io so they may just play nicer together. Honestly it may be easier to just hit the REST API every minute. The devices endpoint will give you the realtime data for each of your devices

[bachya]

@owise1 I took a look at your NodeJS package and it helped me realize that I needed to explicitly set the websocket transport; after that, everything started working. Thank you!