Acceptance test optimization

MobilityData / gtfs-validator

Canonical GTFS Validator project for schedule (static) files.

https://gtfs-validator.mobilitydata.org/

Apache License 2.0

266 stars 100 forks source link

Acceptance test optimization #1698

Closed emmambd closed 1 month ago

emmambd commented 4 months ago

We want to make this process less time-intensive. Suggestions so far:

[ ] Better document filtering
[x] Add performance analysis to the acceptance tests so we catch code changes with memory issues
[x] Remove auto-run acceptance tests in draft mode
[ ] Reuse acceptance test reports when possible. Make sure if the master artifacts are +30 days we run the validator.
[x] Fix warnings in gh action

davidgamez commented 1 month ago

Timebox: 3 days

emmambd commented 1 month ago

@cka-y An out of scope question but asking since it's coming up as I'm reviewing Flex feeds: the performance test is about the amount of time it takes to parse a feed, correct? I'm wondering as well about getting stats on the number of feeds that have unparsable rows grouped by file

cka-y commented 1 month ago

@emmambd -- It's about the amount of time it takes to validate the feed including parsing time. We could look into adding stats aboute the number of feeds with unparsable errors -- shall we open an issue for that?

emmambd commented 1 month ago

@cka-y Can do :) Out of scope for this iteration