`create_transit_net` not producing a complete network from GB Rail GTFS

[double-u-a]

Description of the bug

Incomplete network produced from gtfs data for the GB Rail network. I am assuming something about the gtfs data is causing the issue but urbanaccess did not throw any real errors so I'm not sure. The script below is fairly self contained so it should be easy to replicate. Many thanks for making this module of course!

I also had to fake a 1 row dummy calendar_dates.txt (below) as urbanaccess doesn't run without one present, but none of the feeds I've come across have this file, this may also be causing the issue but I wouldn't know how.

service_id,date,exception_type
FCK,99999999999999999999,2

GTFS feed or OSM data (optional)

http://www.gbrail.info/gtfs.zip

Environment

• Operating system: Ubuntu 16.04

• Python version: 2.7

• UrbanAccess version: Built from github today

Paste the code that reproduces the issue here:

import urbanaccess as ua
import pandas as pd
import pandana as pdna
import urllib

# get gtfs data
gtfs_zip = urllib.URLopener()
gtfs_zip.retrieve("http://www.gbrail.info/gtfs.zip", "gb_rail/gtfs.zip")

# unzip
import zipfile
zip_ref = zipfile.ZipFile('gb_rail/gtfs.zip', 'r')
zip_ref.extractall('gb_rail/gtfs')
zip_ref.close()

# build GTFS dataframe within boundingbox
gtfs_df = ua.gtfs.load.gtfsfeed_to_df(r'gb_rail/gtfs',
                                      validation=False, verbose=False,
                                      bbox=None,
                                      remove_stops_outsidebbox=None,
                                      append_definitions=True)

# build network from GTFS dataframe
tfl_net = ua.gtfs.network.create_transit_net(gtfs_df,
                                   day='monday',
                                   timerange=["07:00:00","19:00:00"],
                                   overwrite_existing_stop_times_int=False,
                                   use_existing_stop_times_int=True)

tfl_net.transit_edges.to_csv(r'gb_rail/network/edge.csv')
tfl_net.transit_nodes.to_csv(r'gb_rail/network/node.csv')

Paste the error message (if applicable):

The process completed without any fatal error messages, with only these two warnings:

sys:1: DtypeWarning: Columns (9,17) have mixed types. Specify dtype option on import or set low_memory=False.
sys:1: DtypeWarning: Columns (9,11) have mixed types. Specify dtype option on import or set low_memory=False.

The green points are the original gtfs/stops.csv, the purple points are what is saved in node.csv from the Urban Access dataframe of the nodes. The edge network is similarly incomplete. I chose a 7 hour time period on a Monday so the majority of the routes should be included.

Missing nodes in green

Incomplete edge network

[sablanchard]

@double-u-a thank you for the issue, we are taking a look at this and will get back to you once we have identified the issue.

[sablanchard]

@double-u-a just want to give you a update: we have identified the issue and are working on the solution and will post back when we are ready for you to test the solution.

[sablanchard]

@double-u-a we solved the issue in this PR #30. The issue was related to the assumption that feeds loaded into urbanaccess would have service_ids and stop_ids that are not shared across agencies. Your feed does in fact share these ids across multiple agencies. We have added support for this gtfs structure type. Can you clone the branch https://github.com/UDST/urbanaccess/tree/support-feeds-wnonunique-relational-ids and test your workflow out and tell us if this fixes your issue?

We have tested your code locally and it looks to be fixed:

[double-u-a]

I can confirm it is outputting a complete network now, thank you very much for solving the issue.

[sablanchard]

Thanks @double-u-a. We will merge into master once we complete our internal testing.

[sablanchard]

fixed with #30

[sablanchard]

@double-u-a just wanted to let you know we have just added a demo for UrbanAccess that shows you an entire workflow in a jupyter notebook that can be accessed here if you are interested.