Closed jdevroom closed 1 year ago
Could you share the exact subscription message you are sending and the exact sample code you are using ?
I tested our python example (https://github.com/aisstream/example/blob/main/python/main_mmsi_message_filter.py) from our examples repo and did not receive the error as well as tested the example from the docs.
I'm not sure why, but the adapted script works now. Thank you for your response anyway.
I struggle with this, here is how I've adapted my code to handle it:
nr_of_restarts = 0
async for websocket in websockets.connect("wss://stream.aisstream.io/v0/stream"):
log.info(f"Running loop, restart nr {nr_of_restarts}")
subscribe_message = ...
subscribe_message_json = json.dumps(subscribe_message)
try:
await websocket.send(subscribe_message_json)
...
except websockets.exceptions.ConnectionClosedError as e:
log.info(f"Caught {e}, restarting")
nr_of_restarts += 1
continue
Log output when catching error is:
Jul 11 12:36:34 shiptracker python[468720]: Caught sent 1011 (unexpected error) keepalive ping timeout; no close frame received, restarting Jul 11 12:36:34 shiptracker python[468720]: Running loop, restart nr 421 Jul 11 12:36:34 shiptracker python[468720]: Awaiting connections... Jul 11 12:38:00 shiptracker python[468720]: Caught sent 1011 (unexpected error) keepalive ping timeout; no close frame received, restarting Jul 11 12:38:00 shiptracker python[468720]: Running loop, restart nr 422 Jul 11 12:38:00 shiptracker python[468720]: Awaiting connections... Jul 11 12:40:17 shiptracker python[468720]: Caught sent 1011 (unexpected error) keepalive ping timeout; no close frame received, restarting
Right now my app is catching this error once every 2 minutes as you can see from output snippet so this doesn't seem to be a very good solution for the long term.
Could you provide your subscription json as well, I will need it to see if the issue is traced to it.
Where are you reading the websockets contents ? We do close connections that are not reading data quick enough.
async for websocket in websockets.connect("wss://stream.aisstream.io/v0/stream"):
log.info(f"Running loop, restart nr {nr_of_restarts}")
subscribe_message = {"APIKey": "...", "BoundingBoxes": [[[-180, -90], [180, 90]]]}
subscribe_message_json = json.dumps(subscribe_message)
try:
await websocket.send(subscribe_message_json)
log.info("Awaiting connections...")
async for message_json in websocket:
message = json.loads(message_json)
message_type = message["MessageType"]
if message_type == "PositionReport":
ais_message = message['Message']['PositionReport']
ship_id = ais_message["UserID"]
lat = ais_message["Latitude"]
lon = ais_message["Longitude"]
sog = ais_message["Sog"]
cog = ais_message["Cog"]
true_heading = ais_message["TrueHeading"]
cur = conn.cursor()
cur.execute(f"SELECT id FROM ships WHERE ship_id = {ship_id};")
if cur.rowcount == 1:
cur.execute(f"""
INSERT INTO AISData
(ship, timestamp, latitude, longitude, sog, cog, true_heading)
VALUES (
(SELECT id FROM ships WHERE ship_id = {ship_id}),
current_timestamp,
{lat}, {lon}, {sog}, {cog}, {true_heading}
)
on conflict do nothing;
""")
if cur.rowcount == 1:
conn.commit()
cur.close()
if message_type == "ShipStaticData":
ship_static = message['Message']['ShipStaticData']
ship_name = ship_static["Name"]
ship_id = ship_static["UserID"]
ship_imo = ship_static["ImoNumber"]
ship = ships.filter(pl.col("IMO_nr") == ship_imo)
if ship.shape[0]:
if df.filter(pl.col("ship_id") == ship_id).shape[0] == 0:
cur = conn.cursor()
cur.execute("INSERT INTO ships (ship_id, ship_name) VALUES (%s, %s) on conflict do nothing;", (ship_id, ship_name))
df_temp = pl.DataFrame({"ship_id": ship_id, "IMO_nr": ship_imo, "ship_name": ship_name})
df = pl.concat([df, df_temp], how="diagonal", rechunk=True)
log.info(f"Now tracking {ship_name}")
log.info(f"Total ships tracked: {df.shape[0]}")
conn.commit()
cur.close()
except websockets.exceptions.ConnectionClosedError as e:
log.info(f"Caught {e}, restarting")
nr_of_restarts += 1
continue
Do you want the API key here or in private?
It seems I've allocated too low CPU and RAM to my VM for this and it keeps maxing out so it may very well be that this is all on me
I am getting ConnectionResetError's with the example Python scripts, and this from C# WebSockets: AIS Stream Error (WebSocketException): Unable to connect to the remote server
.
@jay-tux Could your share your python version ?
3.10 from Arch repos - It seems to be fixed by just retrying a bunch of times
Our C# app always has to retry at least one, but usually after <5 tries it can connect.
Interesting @jay-tux , I would like to get to the bottom of why you have to do that. Are you using the same github account as you are making this comment from. I would like to check the logs.
If you don't mind I may move this to a separate issue.
I think it is the same account @aisstream - go ahead with a different issue but kindly please do ping me as I'm curious as well
Are you still experiencing this issue? I believe it was related to a misconfigured rate limiting mechanism at the load balancer level.
Seems to be fixed indeed. I tried it earlier and got a connection immediately. Thanks for the support
I believe this issue is now fixed. Please re-open if it persists.
Dear, I've just generated an API key and tried to run this python example using my API key: https://aisstream.io/documentation#Python
The default example works, but as soon as I change the MMSI number or bounding box, I get ConnectionResetError returned.
Help welcome, thank you in advance!
Joris