Prevent listing in web-validator public bucket

MobilityData / gtfs-validator

Canonical GTFS Validator project for schedule (static) files.

https://gtfs-validator.mobilitydata.org/

Apache License 2.0

288 stars 101 forks source link

Prevent listing in web-validator public bucket #1796

Closed davidgamez closed 3 months ago

davidgamez commented 3 months ago

Describe the bug

As reported here, the public GCP bucket allows listing.

Steps/Code to Reproduce

Go to https://gtfs-validator-results.mobilitydata.org/ .

Expected Results

Listing operation should be blocked.

Actual Results

The XML listing content is returned

Screenshots

No response

Files used

No response

Validator version

Web-validator(all versions)

Operating system

Web

Java version

No response

Additional notes

No response

qcdyx commented 3 months ago

staging env - not publicly accesible

emmambd commented 3 months ago

Just a thought - is it possible for us to include some kind of "error response" on this page, prompting users to go back to gtfs-validator.mobilitydata.org?

davidgamez commented 3 months ago

This fix intends to block the listing action but allow get actions. If the bucket is fully private, the app stops working. Currently, staging is returning 403 Forbidden when accessing files.

https://github.com/user-attachments/assets/ffbdd60a-2fb0-45f1-afa3-df1657d35f7e

emmambd commented 3 months ago

@davidgamez So "success" is basically what @qcdyx's implemented, then? Is it possible to have a second issue for actually making this page descriptive?

davidgamez commented 3 months ago

@davidgamez So "success" is basically what @qcdyx's implemented, then? Is it possible to have a second issue for actually making this page descriptive?

What is implemented in staging blocks users from using the web validator. We need to allow retrieval of files from the bucket and block just listing operations.

qcdyx commented 3 months ago

Hello @emmambd, @davidgamez and I created a new access role "Mobility Storage Get" in the stg-gtfs-validator-results GCP bucket. Could you try validating a GTFS file in staging https://gtfs-validator-staging.mobilitydata.org/? If it works, I'll apply the same change in prod.

emmambd commented 3 months ago

@qcdyx When I try to validate a feed on staging to generate the results page, the feed just never validates.

davidgamez commented 3 months ago

@emmambd Please try again; it should be OK now.

emmambd commented 3 months ago

Looks good to me! Access now denied without listing.

qcdyx commented 3 months ago

Access denied without listing on Prod. Ready to be closed.