GTFS feed won't load to df (calendar_dates file instead of calendar file)

MartijnVerhoeven commented 5 years ago

Hi,

My name is Martijn Verhoeven and I work for the City of Amsterdam. I am currently trying to apply the bay area demo to Amsterdam with a gtfs feed from our local transit authority GVB. However when trying to load the feed into a dataframe it gives the following error:

'ValueError: calendar.txt is a required GTFS tekst file and was not found in folder'

The feed does have a calendar_dates.txt file however, but I don't know how to apply the parameter calendar_dates_lookup=None to loading a feed to a dataframe.

Do you have any suggestions or python scripts to sort this out?

Kind regards,

Martijn Verhoeven

sablanchard commented 5 years ago

Hi @MartijnVerhoeven , No problem, you can use the script we use for the full integration tests for San Diego for your case so see the file: https://github.com/UDST/urbanaccess/blob/dev/urbanaccess/tests/integration/integration_sandiego.py

Note: 1) until a future update removes the calendar.txt requirement, you should create a dummy file to bypass this issue by doing the following: https://github.com/UDST/urbanaccess/blob/dev/urbanaccess/tests/integration/integration_sandiego.py#L22-L33

2) then you should examine your calendar_dates.txt file and see what columns and values you want to use to select trips from that match your desired operational schedule.

For instance see this example for San Diego in the same file above: https://github.com/UDST/urbanaccess/blob/dev/urbanaccess/tests/integration/integration_sandiego.py#L49-L54
Note the parameter: calendar_dates_lookup={'exception_note': ['FINAL', 'WD'], 'schedule_type': 'WD'} What this is doing is using the column exception_note and its getting all the service ids that have the values in the list: 'FINAL', 'WD' (in the case of San Diego this indicates final schedule version and weekday) and also get service ids that that have column schedule_type value of WD (in the case of San Diego this indicates weekday).

You should be all set after this. We have a more elegant solution for this planned in the pipeline for a future version but have not yet had time to implement it properly yet but this should suffice.

Ill keep this open until you can report back that you successfully got it to work for you.

MartijnVerhoeven commented 5 years ago

Hi Sam,

Thank you for getting back to me quickly. I’ve succesfully created a dummy file and set the parameter. I have a follow up question about the plotted image. Is it possible to export the output of the script to a point shapefile with the corresponding attribute values? If so, do you perhaps have a code snippet I could try.

Kind regards,

Martijn Verhoeven

Data-Analist Team Beeld & Data Ruimte & Duurzaamheid City of Amsterdam

Tel: +31622162693

Werkdagen: Ma t/m Vr

Van: sablanchard [mailto:notifications@github.com] Verzonden: vrijdag 28 juni 2019 17:49 Aan: UDST/urbanaccess CC: Verhoeven, Martijn; Mention Onderwerp: Re: [UDST/urbanaccess] GTFS feed won't load to df (calendar_dates file instead of calendar file) (#56)

Hi @MartijnVerhoevenhttps://github.com/MartijnVerhoeven , No problem, you can use the script we use for the full integration tests for San Diego for your case so see the file: https://github.com/UDST/urbanaccess/blob/dev/urbanaccess/tests/integration/integration_sandiego.py

Note:

until a future update removes the calendar.txt requirement, you should create a dummy file to bypass this issue by doing the following: https://github.com/UDST/urbanaccess/blob/dev/urbanaccess/tests/integration/integration_sandiego.py#L22-L33
then you should examine your calendar_dates.txt file and see what columns and values you want to use to select trips from that match your desired operational schedule.
- For instance see this example for San Diego in the same file above: https://github.com/UDST/urbanaccess/blob/dev/urbanaccess/tests/integration/integration_sandiego.py#L49-L54
- Note the parameter: calendar_dates_lookup={'exception_note': ['FINAL', 'WD'], 'schedule_type': 'WD'} What this is doing is using the column exception_note and its getting all the service ids that have the values in the list: 'FINAL', 'WD' (in the case of San Diego this indicates final schedule version and weekday) and also get service ids that that have column schedule_type value of WD (in the case of San Diego this indicates weekday).

You should be all set after this. We have a more elegant solution for this planned in the pipeline for a future version but have not yet had time to implement it properly yet but this should suffice.

Ill keep this open until you can report back that you successfully got it to work for you.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/UDST/urbanaccess/issues/56?email_source=notifications&email_token=AL6QMQSVCCMZ3QVHK5ID65DP4YXIHA5CNFSM4H34JUH2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODY2OJ7A#issuecomment-506782972, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AL6QMQRKLGITOLLAZDMRPL3P4YXIHANCNFSM4H34JUHQ.

De gemeente Amsterdam streeft naar optimale dienstverlening en zorgvuldige afhandeling van e-mailverkeer. Als een e-mail niet voor u is bestemd, verzoekt de gemeente u vriendelijk ons van de juiste adressering op de hoogte te stellen en de e-mail te verwijderen zonder de informatie te gebruiken en te delen met anderen. Voor verdere informatie over de rechten op informatie, zie https://www.amsterdam.nl/proclaimer.

sablanchard commented 5 years ago

Great to hear @MartijnVerhoeven ! Will close this issue then.

Yes you could export your nodes to a shapefile using something like this:

import geopandas as gpd
import shapely

point_gdf = gpd.GeoDataFrame(nodes_df, crs={'init': 'epsg:4326'},
                             geometry=[shapely.geometry.Point(xy) for xy in zip(nodes_df.x, nodes_df.y)])
point_gdf.to_file('net_nodes.shp')

UDST / urbanaccess

GTFS feed won't load to df (calendar_dates file instead of calendar file) #56