search

jamespfennell / path-train-gtfs-realtime

GTFS Realtime for the PATH train in New York and New Jersey
MIT License
12 stars 3 forks source link

Hosted data feeds appear to be having issues #7

Open cedarbaum opened 8 months ago

cedarbaum commented 8 months ago

It seems that both https://path.transitdata.nyc/gtfsrt and https://path2.transitdata.nyc/gtfsrt feeds are having intermittent issues right now (either returning incomplete or empty proto data). Running the application locally still works with the --use_panynj_api option. @jamespfennell when you have time, could you see if there are any obvious issues in the logs? If the issue is resolved by the time you check, please feel free to close.

Note also that I am seeing some incomplete data on the path-data demo site as well (https://jsfiddle.net/qkp7g8ze/embedded/result/), but I would expect the PANYNJ API to be working (assuming that is what is running on path2).

jamespfennell commented 8 months ago

Hey, I don't have time to debug fully right now but I'm definitely seeing in my monitoring that both applications have been crashing throughout the day. It seems to have started today. Unfortunately my deployment environment is such that I lose the logs when the Docker container crashes (oops), so I don't have log lines from the crashes. I can fix the logging setup to capture the logs externally and see if I can capture a crash.

Other services running on my VM don't seem affected, and the VM logging (e.g RAM usage) look totally normal, so I don't think it's a problem with the VM I'm running this on.

cedarbaum commented 7 months ago

Hey @jamespfennell - one other issue I am seeing is that I think both https://path.transitdata.nyc/ and https://path2.transitdata.nyc/ seem to be returning the same data now. My guess is that both are using the mrazza API. It's possible I am doing something incorrect on my end, but is it possible there is a config issue here on the server?

jamespfennell commented 7 months ago

Woops! My reverse proxy config was pointing both URLs to the same deployment of this app. Just fixed it!

jamespfennell commented 7 months ago

I guess it would be nice if the html status page (https://path.transitdata.nyc/) showed which flags are being used? Sorry for the hassle!

cedarbaum commented 7 months ago

Thanks for taking a look and the quick fix!

Also, feel free to close this issue if things look OK on your end. I think the services are running more reliably now from my end (aside from the known intermittent issues with the mrazza API, which I think is a separate issue).

I wasn't sure if you're using this to track your investigation into the previous crashes and/or the logging improvements you mentioned, which I why I haven't closed it myself.