jarondl / pygtfs

A python (2/3) library for GTFS
MIT License
63 stars 44 forks source link

Import fails when conditionally required route.agency_id is missing #62

Open Bertware opened 3 years ago

Bertware commented 3 years ago

When agency_id is not set on a route, the import fails. Example feed: https://www.bart.gov/sites/default/files/docs/google_transit_20211001_20220213_v1.zip

Traceback (most recent call last):

  File "/home/travis/virtualenv/python3.9.1/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1779, in _execute_context

    self.dialect.do_executemany(

  File "/home/travis/virtualenv/python3.9.1/lib/python3.9/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py", line 962, in do_executemany

    context._psycopg2_fetched_rows = xtras.execute_values(

  File "/home/travis/virtualenv/python3.9.1/lib/python3.9/site-packages/psycopg2/extras.py", line 1270, in execute_values

    cur.execute(b''.join(parts))

psycopg2.errors.ForeignKeyViolation: insert or update on table "routes" violates foreign key constraint "routes_feed_id_fkey"

DETAIL:  Key (feed_id, agency_id)=(1, None) is not present in table "agency".

From the spec:

Field Name Type Presence Description
agency_id ID referencing agency.agency_id Conditionally Required Agency for the specified route.Conditionally Required:- Required if multiple agencies are defined in agency.txt. - Optional otherwise.

Suggested fix: set the agency_id automatically to the one included agency if no agency_id is defined

jarondl commented 3 years ago

Where do you want to set the id automatically? in the agency table?

If you want to send a PR, I'll be happy to review and merge.

Bertware commented 3 years ago

This issue is not directly affecting me, I merely noticed it while writing another PR, so I figured I'd report it in case anyone else wants to take it up.

The agency id would be set automatically in the routes table, and it would point to the only agency that is present in the agencies table. If multiple IDs is set, agency_id becomes required on routes.txt.