search

cal-itp / data-infra

Cal-ITP data infrastructure
https://docs.calitp.org/data-infra
GNU Affero General Public License v3.0
48 stars 13 forks source link

ZLib input stream exceptions occurring in RT pipeline #2608

Open cal-itp-sentry[bot] opened 1 year ago

cal-itp-sentry[bot] commented 1 year ago

Sentry Issue: CAL-ITP-DATA-INFRA-24J2

CalledProcessError: Command '['java', '-jar', '/gtfs-realtime-validator.jar', '-gtfs', '/tmp/tmpojtth514/google_transit.zip', '-gtfsRealtimePath', '/tmp/tmpojtth514/rt_44a2869ad642dfa9a626ddd7995e0393/', '-sort', 'name']' returned non-zero exit status 1.
  File "gtfs_rt_parser.py", line 634, in parse_and_validate
    return validate_and_upload(
  File "gtfs_rt_parser.py", line 377, in validate_and_upload
    execute_rt_validator(
  File "gtfs_rt_parser.py", line 360, in execute_rt_validator
    subprocess.run(

Specific event link: https://sentry.calitp.org/organizations/sentry/issues/70478/events/deaf455044554b018371ba98db2a1946/?project=2&referrer=events-table%2F%3Fproject%3D2

This specific event is a ZLib input stream error. First step is reproducing the issue, and determining if this is something we need to handle (i.e. we need a retry, or there's a file system issue) or if we need to raise this up to the RT validator repo.

lauriemerrell commented 1 year ago

Another example issue: https://sentry.calitp.org/organizations/sentry/issues/72508/