search

OneBusAway / onebusaway-application-modules

The core OneBusAway application suite.
https://github.com/OneBusAway/onebusaway-application-modules/wiki
Other
212 stars 133 forks source link

/vehicles-for-agency/agency_id.json is empty #113

Closed luqmaan closed 9 years ago

luqmaan commented 9 years ago

I am setting up a OneBusAway instance, but can't figure out how to get /vehicles-for-agency to respond with the vehicles from my GTFS-RT Vehicle Positions feed.

Am I configuring OBA incorrectly? Is the PB feed I am using invalid?

Running a OneBusAway 1.1.13 instance using the data from:

The GTFS zip file provided by CapMetro has a null agency_id, so I use bundle.xml to set it to 666

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd">
    <bean id="gtfs-bundles" class="org.onebusaway.transit_data_federation.bundle.model.GtfsBundles">
        <property name="bundles">
            <list>
                <ref bean="gtfsCapMetro" />
            </list>
        </property>
    </bean>

    <bean id="gtfsCapMetro" class="org.onebusaway.transit_data_federation.bundle.model.GtfsBundle">
        <property name="path" value="/Users/luqmaan/dev/capmetrobeta/gtfs.zip" />
        <property name="defaultAgencyId" value="666" />
    </bean>
</beans>
java -Xmx1G -server -jar oba_full.war -build bundle.xml gtfs-bundle \
    -P tripEntriesFactory.throwExceptionOnInvalidStopToShapeMappingException=false \
    -P distanceAlongShapeLibrary.localMinimumThreshold=50 \
    -P distanceAlongShapeLibrary.maxDistanceFromStopToShapePoint=1000

I start the OBA full webapp and specify the gtfsRealtimeVehiclePositionsUrl:

java -jar oba_full.war \
    -webapp \
    -gtfsRealtimeVehiclePositionsUrl='https://data.texas.gov/api/file_data/FIMhW19BoaBhaoCIZSG3G5CGSx2KiAH7TDJI2KRB-ms?filename=VehLoc.pb' \
    gtfs-bundle

The webapp starts and I see GTFS REALTIME! logged.

I can visit http://localhost:8080/api/where/agencies-with-coverage.json?key=web and get this response back:

{
    "code": 200,
    "currentTime": 1421280159889,
    "data": {
        "limitExceeded": false,
        "list": [
            {
                "agencyId": "666",
                "lat": 30.366932,
                "latSpan": 0.4393599999999971,
                "lon": -97.6807445,
                "lonSpan": 0.620711
            }
        ],
        "references": {
            "agencies": [
                {
                    "disclaimer": "",
                    "id": "666",
                    "lang": "en",
                    "name": "Capital Metro",
                    "phone": "512-474-1200",
                    "privateService": false,
                    "timezone": "America/Chicago",
                    "url": "http://www.capmetro.org"
                }
            ],
            "routes": [],
            "situations": [],
            "stops": [],
            "trips": []
        }
    },
    "text": "OK",
    "version": 2
}

But when I visit http://localhost:8080/api/where/vehicles-for-agency/666.json?key=web, I see an empty response:

{
    "code": 200,
    "currentTime": 1421280274313,
    "data": {
        "limitExceeded": false,
        "list": [],
        "outOfRange": false,
        "references": {
            "agencies": [],
            "routes": [],
            "situations": [],
            "stops": [],
            "trips": []
        }
    },
    "text": "OK",
    "version": 2
}

The rest of the OBA API endpoints respond with static data from the GTFS file, e.g. http://localhost:8080/api/where/arrivals-and-departures-for-stop/666_2643.json?key=web or http://localhost:8080/api/where/trips-for-route/666_801.json?key=web.

I don't see anything wrong with the Vehicle Position PB feed I am consuming. Here's a dump from gtfs-rt-dump:

header {
  gtfs_realtime_version: "1.0"
  timestamp: 1421190223
}
entity {
  id: "1"
  vehicle {
    trip {
      trip_id: "1392240"
      start_time: "17:02"
      start_date: "20150113"
      route_id: "550"
    }
    position {
      latitude: 30.478926
      longitude: -97.77894
      speed: 20.28
    }
    vehicle {
      id: "11105"
    }
  }
}
entity {
  id: "2"
  vehicle {
    trip {
      trip_id: "1392239"
      start_time: "17:02"
      start_date: "20150113"
      route_id: "550"
    }
    position {
      latitude: 30.268137
      longitude: -97.715324
      speed: 24.95
    }
    vehicle {
      id: "11101"
    }
  }
}
entity {
  id: "3"
  vehicle {
    trip {
      trip_id: "1400437"
      start_time: "17:02"
      start_date: "20150113"
      route_id: "801"
    }
    position {
      latitude: 30.349812
      longitude: -97.71127
      speed: 4.34
    }
    vehicle {
      id: "5011"
    }
  }
}
entity {
  id: "4"
  vehicle {
    trip {
      trip_id: "1400436"
      start_time: "17:02"
      start_date: "20150113"
      route_id: "801"
    }
    position {
      latitude: 30.336668
      longitude: -97.719505
      speed: 0.62
    }
    vehicle {
      id: "5018"
    }
  }
}
entity {
  id: "5"
  vehicle {
    trip {
      trip_id: "1400435"
      start_time: "17:02"
      start_date: "20150113"
      route_id: "801"
    }
    position {
      latitude: 30.310543
      longitude: -97.73312
      speed: 19.26
    }
    vehicle {
      id: "5019"
    }
  }
}
entity {
  id: "6"
  vehicle {
    trip {
      trip_id: "1400434"
      start_time: "17:02"
      start_date: "20150113"
      route_id: "801"
    }
    position {
      latitude: 30.266378
      longitude: -97.744965
      speed: 6.11
    }
    vehicle {
      id: "5016"
    }
  }
}
entity {
  id: "7"
  vehicle {
    trip {
      trip_id: "1400433"
      start_time: "17:02"
      start_date: "20150113"
      route_id: "801"
    }
    position {
      latitude: 30.258127
      longitude: -97.74785
      speed: 11.01
    }
    vehicle {
      id: "5014"
    }
  }
}
entity {
  id: "8"
  vehicle {
    trip {
      trip_id: "1400432"
      start_time: "17:02"
      start_date: "20150113"
      route_id: "801"
    }
    position {
      latitude: 30.222488
      longitude: -97.766464
      speed: 2.93
    }
    vehicle {
      id: "5013"
    }
  }
}
entity {
  id: "9"
  vehicle {
    trip {
      trip_id: "1400496"
      start_time: "17:01"
      start_date: "20150113"
      route_id: "801"
    }
    position {
      latitude: 30.41848
      longitude: -97.66868
    }
    vehicle {
      id: "5012"
    }
  }
}
entity {
  id: "10"
  vehicle {
    trip {
      trip_id: "1400444"
      start_time: "17:01"
      start_date: "20150113"
      route_id: "801"
    }
    position {
      latitude: 30.162895
      longitude: -97.790436
    }
    vehicle {
      id: "5008"
    }
  }
}
entity {
  id: "11"
  vehicle {
    trip {
      trip_id: "1400438"
      start_time: "17:02"
      start_date: "20150113"
      route_id: "801"
    }
    position {
      latitude: 30.404785
      longitude: -97.67872
      speed: 16.56
    }
    vehicle {
      id: "5015"
    }
  }
}
entity {
  id: "12"
  vehicle {
    trip {
      trip_id: "1400431"
      start_time: "17:02"
      start_date: "20150113"
      route_id: "801"
    }
    position {
      latitude: 30.18913
      longitude: -97.780174
      speed: 48.25
    }
    vehicle {
      id: "5010"
    }
  }
}
entity {
  id: "13"
  vehicle {
    trip {
      trip_id: "1400883"
      start_time: "17:01"
      start_date: "20150113"
      route_id: "803"
    }
    position {
      latitude: 30.3164
      longitude: -97.73578
      speed: 0.07
    }
    vehicle {
      id: "5051"
    }
  }
}
entity {
  id: "14"
  vehicle {
    trip {
      trip_id: "1400882"
      start_time: "17:02"
      start_date: "20150113"
      route_id: "803"
    }
    position {
      latitude: 30.28923
      longitude: -97.741356
      speed: 2.17
    }
    vehicle {
      id: "5064"
    }
  }
}
entity {
  id: "15"
  vehicle {
    trip {
      trip_id: "1400881"
      start_time: "17:02"
      start_date: "20150113"
      route_id: "803"
    }
    position {
      latitude: 30.275562
      longitude: -97.74262
      speed: 0.98
    }
    vehicle {
      id: "5054"
    }
  }
}
entity {
  id: "16"
  vehicle {
    trip {
      trip_id: "1400878"
      start_time: "17:01"
      start_date: "20150113"
      route_id: "803"
    }
    position {
      latitude: 30.229088
      longitude: -97.8
    }
    vehicle {
      id: "5059"
    }
  }
}
entity {
  id: "17"
  vehicle {
    trip {
      trip_id: "1400885"
      start_time: "17:02"
      start_date: "20150113"
      route_id: "803"
    }
    position {
      latitude: 30.39978
      longitude: -97.72039
      speed: 4.69
    }
    vehicle {
      id: "5057"
    }
  }
}
entity {
  id: "18"
  vehicle {
    trip {
      trip_id: "1400884"
      start_time: "17:02"
      start_date: "20150113"
      route_id: "803"
    }
    position {
      latitude: 30.36095
      longitude: -97.72948
      speed: 20.66
    }
    vehicle {
      id: "5060"
    }
  }
}
entity {
  id: "19"
  vehicle {
    trip {
      trip_id: "1400880"
      start_time: "17:02"
      start_date: "20150113"
      route_id: "803"
    }
    position {
      latitude: 30.253948
      longitude: -97.762665
      speed: 24.66
    }
    vehicle {
      id: "5065"
    }
  }
}
entity {
  id: "20"
  vehicle {
    trip {
      trip_id: "1400879"
      start_time: "17:02"
      start_date: "20150113"
      route_id: "803"
    }
    position {
      latitude: 30.233763
      longitude: -97.79636
      speed: 16.91
    }
    vehicle {
      id: "5063"
    }
  }
}
barbeau commented 9 years ago

From a quick glance I don't see any red flags. Do you have access to a TripUpdates feed as well? If so, you might want to feed that in too and see what happens. IIRC OBA needs info internally that's not normally in the VehiclePositions feed.

Can you also post a plain text version of the VehiclePosition feed? On Jan 14, 2015 7:06 PM, "Luqmaan Dawoodjee" notifications@github.com wrote:

I am setting up a OneBusAway instance, but can't figure out how to get /vehicles-for-agency to respond with the vehicles from my GTFS-RT Vehicle Positions feed.

Am I configuring OBA incorrectly? Is the PB feed I am using invalid?

Running a OneBusAway 1.1.13 instance using the data from:

The GTFS zip file provided by CapMetro has a null agency_id, so I use bundle.xml to set it to 666

<?xml version="1.0" encoding="UTF-8"?>

``` ```

java -Xmx1G -server -jar oba_full.war -build bundle.xml gtfs-bundle \ -P tripEntriesFactory.throwExceptionOnInvalidStopToShapeMappingException=false \ -P distanceAlongShapeLibrary.localMinimumThreshold=50 \ -P distanceAlongShapeLibrary.maxDistanceFromStopToShapePoint=1000

I start the OBA full webapp and specify the gtfsRealtimeVehiclePositionsUrl:

java -jar oba_full.war \ -webapp \ -gtfsRealtimeVehiclePositionsUrl='https://data.texas.gov/api/file_data/FIMhW19BoaBhaoCIZSG3G5CGSx2KiAH7TDJI2KRB-ms?filename=VehLoc.pb' \ gtfs-bundle

The webapp starts and I see GTFS REALTIME! logged.

I can visit http://localhost:8080/api/where/agencies-with-coverage.json?key=web and get this response back:

{ "code": 200, "currentTime": 1421280159889, "data": { "limitExceeded": false, "list": [ { "agencyId": "666", "lat": 30.366932, "latSpan": 0.4393599999999971, "lon": -97.6807445, "lonSpan": 0.620711 } ], "references": { "agencies": [ { "disclaimer": "", "id": "666", "lang": "en", "name": "Capital Metro", "phone": "512-474-1200", "privateService": false, "timezone": "America/Chicago", "url": "http://www.capmetro.org" } ], "routes": [], "situations": [], "stops": [], "trips": [] } }, "text": "OK", "version": 2 }

But when I visit http://localhost:8080/api/where/vehicles-for-agency/666.json?key=web, I see an empty response:

{ "code": 200, "currentTime": 1421280274313, "data": { "limitExceeded": false, "list": [], "outOfRange": false, "references": { "agencies": [], "routes": [], "situations": [], "stops": [], "trips": [] } }, "text": "OK", "version": 2 }

— Reply to this email directly or view it on GitHub https://github.com/OneBusAway/onebusaway-application-modules/issues/113.

luqmaan commented 9 years ago

Updated the issue with a gtfs-rt-dump of the VehiclePosition feed. I'll try the Alerts feed, as CapMetro doesn't publish a TripUpdates feed.

kurtraschke commented 9 years ago

Agree with @barbeau; without a TripUpdates feed OneBusAway will discard the vehicle positions. Arguably this is bad behavior, as we could pass the vehicle positions on to downstream API and GTFS-realtime consumers, but in general OneBusAway was built around providing arrival predictions, which don't actually depend on having vehicle positions available.

Aside from that, the VehiclePositions feed looks good aside from the fact that the start_time field in the TripDescriptor instances only provides hours and minutes...the spec requires the format HH:MM:SS. In addition, from the distinct trip IDs I am assuming these are not frequency-expanded trips, in which case the provision of start_time is not required (and technically invalid under the current spec), but as far as OneBusAway is concerned, that's all irrelevant as the current implementation (on the master branch) doesn't use the start_time field anyway.

luqmaan commented 9 years ago

without a TripUpdates feed OneBusAway will discard the vehicle positions

Well that sucks. :crying_cat_face:

Maybe the OBA docs should be updated to indicate that you can't use the VehiclePositions feed without also providing the TripUpdates feed?

Also, thanks for pointing out the deficiencies in the PB file!

barbeau commented 9 years ago

Maybe the OBA docs should be updated to indicate that you can't use the VehiclePositions feed without also providing the TripUpdates feed?

Yeah, that's fair.

@kurtraschke or @sheldonabrown Do you know if this behavior changes with OBA NYC and the SIRI API? In other words, if you put the pieces together to feed in GTFS-rt Vehicle Positions feed such as this one and you hit the SIRI VehicleMonitoring API, would you get a dump of the vehicles? Or would you also get an empty list?

I've been looking towards the SIRI API for vehicle-related info, primarily because to my knowledge the OBA REST API lacks functionality for basic vehicle-related things like retrieving vehicles-per-route.

luqmaan commented 9 years ago

Also, my OBA instance is consuming the Service Alerts feed without any problems:

java -jar ~/dev/capmetrobeta/onebusaway-quickstart-assembly-1.1.13-webapp.war -webapp \
    -gtfsRealtimeAlertsUrl="https://data.texas.gov/api/file_data/WUeIr4_ttCVmqLIPHSe6Wpfg3eN270AWG5gzmMQ9-wo?filename=alert.pb" \
    ~/dev/capmetrobeta/gtfs-bundle

http://localhost:8080/api/where/arrivals-and-departures-for-stop/666_866.json?key=web https://gist.github.com/luqmaan/7ced148a3ff98495b08d

barbeau commented 9 years ago

FYI - use of start_time for non frequency-expanded scheduled trips is now legit: https://developers.google.com/transit/gtfs-realtime/reference#TripDescriptor

Full discussion in: https://groups.google.com/d/msg/gtfs-realtime/qfg1waEgetU/oyPyLoZot0kJ

luqmaan commented 9 years ago

Thank you for the answers, I'm closing this issue since the answer is you can't use vehicles-for-agency without both the Trip Updates and Vehicle Positions feed. I'm waiting on CapMetro to create a Trip Updates feed.

As for vehicles-for-route endpoint, wouldn't you be able to user trips-for-route and filter the trips that have vehicles?

barbeau commented 9 years ago

As for vehicles-for-route endpoint, wouldn't you be able to user trips-for-route and filter the trips that have vehicles?

Yeah, grabbing the trip-status elements from that API call should work - I actually missed that tripStatus elements were included there, and assumed another call to trip-details was necessary. I've also been looking more recently at pulling this from the arrivals-and-departures-for-stop API, if the user is looking in the context of a single stop, since tripStatus is included there too.