ConnectionResetError

[jdevroom]

Dear, I've just generated an API key and tried to run this python example using my API key: https://aisstream.io/documentation#Python

The default example works, but as soon as I change the MMSI number or bounding box, I get ConnectionResetError returned.

Help welcome, thank you in advance!

Joris

[aisstream]

Could you share the exact subscription message you are sending and the exact sample code you are using ?

I tested our python example (https://github.com/aisstream/example/blob/main/python/main_mmsi_message_filter.py) from our examples repo and did not receive the error as well as tested the example from the docs.

[jdevroom]

I'm not sure why, but the adapted script works now. Thank you for your response anyway.

[Olarm]

I struggle with this, here is how I've adapted my code to handle it:

    nr_of_restarts = 0
    async for websocket in websockets.connect("wss://stream.aisstream.io/v0/stream"):
        log.info(f"Running loop, restart nr {nr_of_restarts}")
        subscribe_message = ...
        subscribe_message_json = json.dumps(subscribe_message)

        try:
            await websocket.send(subscribe_message_json)
            ...
        except websockets.exceptions.ConnectionClosedError as e:
            log.info(f"Caught {e}, restarting")
            nr_of_restarts += 1
            continue

Log output when catching error is:

Jul 11 12:36:34 shiptracker python[468720]: Caught sent 1011 (unexpected error) keepalive ping timeout; no close frame received, restarting Jul 11 12:36:34 shiptracker python[468720]: Running loop, restart nr 421 Jul 11 12:36:34 shiptracker python[468720]: Awaiting connections... Jul 11 12:38:00 shiptracker python[468720]: Caught sent 1011 (unexpected error) keepalive ping timeout; no close frame received, restarting Jul 11 12:38:00 shiptracker python[468720]: Running loop, restart nr 422 Jul 11 12:38:00 shiptracker python[468720]: Awaiting connections... Jul 11 12:40:17 shiptracker python[468720]: Caught sent 1011 (unexpected error) keepalive ping timeout; no close frame received, restarting

Right now my app is catching this error once every 2 minutes as you can see from output snippet so this doesn't seem to be a very good solution for the long term.

[aisstream]

Could you provide your subscription json as well, I will need it to see if the issue is traced to it.

Where are you reading the websockets contents ? We do close connections that are not reading data quick enough.

[Olarm]

    async for websocket in websockets.connect("wss://stream.aisstream.io/v0/stream"):
        log.info(f"Running loop, restart nr {nr_of_restarts}")
        subscribe_message = {"APIKey": "...", "BoundingBoxes": [[[-180, -90], [180, 90]]]}
        subscribe_message_json = json.dumps(subscribe_message)

        try:
            await websocket.send(subscribe_message_json)
            log.info("Awaiting connections...")

            async for message_json in websocket:
                message = json.loads(message_json)
                message_type = message["MessageType"]
                if message_type == "PositionReport":
                    ais_message = message['Message']['PositionReport']
                    ship_id = ais_message["UserID"]
                    lat = ais_message["Latitude"]
                    lon = ais_message["Longitude"]
                    sog = ais_message["Sog"]
                    cog = ais_message["Cog"]
                    true_heading = ais_message["TrueHeading"]
                    cur = conn.cursor()
                    cur.execute(f"SELECT id FROM ships WHERE ship_id = {ship_id};")
                    if cur.rowcount == 1:
                        cur.execute(f"""
                            INSERT INTO AISData 
                                (ship, timestamp, latitude, longitude, sog, cog, true_heading)
                            VALUES (
                                (SELECT id FROM ships WHERE ship_id = {ship_id}),
                                current_timestamp,
                                {lat}, {lon}, {sog}, {cog}, {true_heading}
                            )
                            on conflict do nothing;
                        """)
                        if cur.rowcount == 1:
                            conn.commit()
                    cur.close()

                if message_type == "ShipStaticData":
                    ship_static = message['Message']['ShipStaticData']
                    ship_name = ship_static["Name"]
                    ship_id = ship_static["UserID"]
                    ship_imo = ship_static["ImoNumber"]
                    ship = ships.filter(pl.col("IMO_nr") == ship_imo)
                    if ship.shape[0]:
                        if df.filter(pl.col("ship_id") == ship_id).shape[0] == 0:
                            cur = conn.cursor()
                            cur.execute("INSERT INTO ships (ship_id, ship_name) VALUES (%s, %s) on conflict do nothing;", (ship_id, ship_name))
                            df_temp = pl.DataFrame({"ship_id": ship_id, "IMO_nr": ship_imo, "ship_name": ship_name})
                            df = pl.concat([df, df_temp], how="diagonal", rechunk=True)
                            log.info(f"Now tracking {ship_name}")
                            log.info(f"Total ships tracked: {df.shape[0]}")
                            conn.commit()
                            cur.close()
        except websockets.exceptions.ConnectionClosedError as e:
            log.info(f"Caught {e}, restarting")
            nr_of_restarts += 1
            continue

Do you want the API key here or in private?

[Olarm]

It seems I've allocated too low CPU and RAM to my VM for this and it keeps maxing out so it may very well be that this is all on me

[jay-tux]

I am getting ConnectionResetError's with the example Python scripts, and this from C# WebSockets: AIS Stream Error (WebSocketException): Unable to connect to the remote server.

[aisstream]

@jay-tux Could your share your python version ?

[jay-tux]

3.10 from Arch repos - It seems to be fixed by just retrying a bunch of times

Our C# app always has to retry at least one, but usually after <5 tries it can connect.

[aisstream]

Interesting @jay-tux , I would like to get to the bottom of why you have to do that. Are you using the same github account as you are making this comment from. I would like to check the logs.

If you don't mind I may move this to a separate issue.

[jay-tux]

I think it is the same account @aisstream - go ahead with a different issue but kindly please do ping me as I'm curious as well

[aisstream]

Are you still experiencing this issue? I believe it was related to a misconfigured rate limiting mechanism at the load balancer level.

[jay-tux]

Seems to be fixed indeed. I tried it earlier and got a connection immediately. Thanks for the support

[aisstream]

I believe this issue is now fixed. Please re-open if it persists.