MobilityData / gtfs-validator

Canonical GTFS Validator project for schedule (static) files.
https://gtfs-validator.mobilitydata.org/
Apache License 2.0
278 stars 100 forks source link

"load from url" fails with Granicus Website #1813

Open vevetron opened 1 week ago

vevetron commented 1 week ago

Describe the bug

Attempts to validated: Calabasas but get "Error Processing Report".

I don't see anything in the inspection that suggests an error.

But I think what could be happening, Granicus probably blocks requests from cloud servers. So we put in the url, mobility-data server puts in a request for the file, it gets blocked, and we get an error.

It's okay if you download the file and upload it directly.

Steps/Code to Reproduce

Go here: https://gtfs-validator.mobilitydata.org/

Put in this url: Calabasas to "Load from a URL"

Expected Results

Should process the gtfs

Actual Results

"Error Processing Result"

Screenshots

No response

Files used

No response

Validator version

Can't tell - 9/9/2024 version

Operating system

Windows - Chrome

Java version

No response

Additional notes

No response

welcome[bot] commented 1 week ago

Thanks for opening your first issue in this project! If you haven't already, you can join our slack and join the #gtfs-validators channel to meet our awesome community. Come say hi :wave:!

Welcome to the community and thank you for your engagement in open source! :tada:

emmambd commented 1 week ago

Hi @vevetron - thanks for flagging this! We tested this and saw this notice from the Granicus website in our logs:

Access Denied You don't have permission to access "http://www.cityofcalabasas.com/home/showpublisheddocument/31620/638611519891730000" on this server. Reference #18.9369dc17.1725907184.2f7ad1eb https://errors.edgesuite.net/18.9369dc17.1725907184.2f7ad1eb

We suspect this may because our user agent is blocked by the website. The user agent we provide is shared here.

We'd suggest troubleshooting this on the Granicus website to verify if this is the issue.

Let us know if there's anything else we can do to support with this problem.

vevetron commented 1 week ago

We saw something similar since we made requests from Google Cloud. Options: