search

cooltrain7 / Planetside-2-API-Tracker

An unofficial repository for Planetside 2 (PS2) Census API Examples, Issues, and Wiki. Collected and Maintained by members of the PS2 API Community.
http://census.daybreakgames.com/
33 stars 8 forks source link

ECONNRESET responses from Census #12

Open Maelstromeous opened 3 years ago

Maelstromeous commented 3 years ago

Occasionally, Census REST responds with nonsence or a ECONNRESET (also known as a peer connection reset). In my very detailed monitoring, I've calculated on average this happens is ~50 a day.

When it does happen, it appears to happen for multiple requets being made to a particular endpoint, or all at once. In example, it behaves in the following ways:

1) Requests go through as normal 2) The issue occurs 3) All requests on all endpoints respond with ECONNRESET for ~2 minutes 4) Requests return back to normal until it happens again.

It acts like it's being rate limited. However in having chats with Alvin (API maintainer) my serviceid hasn't hit such limits.

DBG staff are struggling to figure out when this occurs, even after providing them very detailed metrics (such as below). All times are in UTC.

For context, my application uses the NodeJS Axios library, but after extensive googling I could find nothing to support evidence that the package has any issues of this manner. This is purely connection issue on the Census side.

Screenshot 2021-02-03 at 20 56 35

dbrennand commented 2 years ago

Interesting. I'm monitoring https://census.daybreakgames.com/ with Uptime-Kuma and I'm also seeing these ECONNRESET responses. Currently, it checks every 2 minutes.

Screenshot 2022-02-23 110149

Screenshot 2022-02-23 110326

Uptime-Kuma's HTTP(S) monitoring also uses axios.

It would be interesting if anyone else gets this error using another library. In the meantime, I'm going to reduce the check to every minute.

Maelstromeous commented 2 years ago

I'm using axios and I'm getting this pretty much 10-20 times an hour. I'm also a heavy Census user so I'm going to see more errors naturally.

fredkilbourn commented 2 years ago

I'm using got and get them periodically, but I don't log them all so not sure how often.

microwavekonijn commented 1 year ago

Probably good to add some context. This was due to the cache not working as suppose to and thus the making more requests then necessary to Census.

Maelstromeous commented 1 year ago

While yes that is true, the responses from Census are rather shall we say... undescriptive to indicate either rate limiting or anything to indicate you're hammering it.

microwavekonijn commented 1 year ago

While I do agree, I would like to see use of headers and status codes, I think the thing is it just get overwhelmed and this is the only thing it can do.