search

openmobilityfoundation / mobility-data-specification

A data standard to enable right-of-way regulation and two-way communication between mobility companies and local governments.
https://www.openmobilityfoundation.org/about-mds/
Other
676 stars 232 forks source link

Feature modes passenger services #763

Closed schnuerle closed 1 year ago

schnuerle commented 2 years ago

Explain pull request

Work on passenger services to incorporate into Modes

Is this a breaking change

Impacted Spec

Which spec(s) will this pull request impact?

Additional context

Working document to synthesize level data US cities are already getting from ride hail companies:

https://docs.google.com/document/d/1OV6MiH0bZSF_jswCNwObKwXO-PYdr_uWxf3no27Slk4/edit#

alexdemisch commented 2 years ago

Comparison to SFMTA Taxi Trips & Telemetry API

I'm listing below fields from SFMTA's current Taxi Trips & Telemetry API and, where applicable, suggestions on how they could be mapped to the passenger-services trips endpoint and/or my thoughts on how to support them. There’s definitely more in the SFMTA spec that what the current passenger-services spec calls for. We might be able to use vehicle_attributes and trip_attributes to support them. I haven’t considered if/how the state machine might affect this, and these are just initial thoughts – some of this is probably worth a discussion about use cases.

SFMTA Trips API

  1. ProviderID – SFMTA collects dispatch company ID separately from the company actually serving the trip. May need another field to capture this in MDS.
  2. TaxiCompanyID – Use MDS provider_id
  3. VehicleID – VIN of taxicab. May need another way to capture this in MDS. Maybe in vehicle_attributes?
  4. VehiclePlacardNumber – Actual number painted on the taxicab, i.e., medallion ID. Can likely use MDS vehicle_id.
  5. LicensePlate - May need another way to capture this in MDS. Maybe in vehicle_attributes?
  6. TripID – Use MDS trip_id
  7. HailType – e.g., street hail, name of app used. Trip_attributes is currently not used for passenger-services, but perhaps it could belong here?
  8. OperatorID – drivers license number. Maybe we need a driver attributes field/array?
  9. StartTimeMilliseconds – Use MDS start_time
  10. EndTimeMilliseconds – Use MDS end_time
  11. PickupLocationAddress – string field with address. Could be added as a field, or perhaps something for trip_attributes? May need to discuss use cases.
  12. PickupLocationLatitude – Use MDS route array
  13. PickupLocationLongitude – Use MDS route array
  14. DropOffLocationAddress -- string field with address. Could be added as a field, or perhaps something for trip_attributes? May need to discuss use cases.
  15. DropoffLocationLatitude – Use MDS route array
  16. DropoffLocationLongitude – Use MDS route array
  17. PassengerCount – Utilize journey_id?
  18. IsWheelChairTransported – Perhaps something for trip_attributes?
  19. TotalFareAmount – Spec defines this as “Total fare for the trip, including all tolls, tips, fees, extras, flag drop, and meter amount.” Could use MDS standard_cost
  20. MeterFareAmount – Spec defines this as “Cost to the customer for the trip, as reported by the meter (excluding tips, fees, tolls, extra amounts).” Could use MDS actual_cost
  21. UpfrontPricing – Spec defines this as “Agreed upon rate that should not change based on the meter, close to what the meter would charge. Maybe use trip_attributes?”
  22. PromoRate – e.g., Yellow Cab has $35 SFO flat rate. Maybe use trip_attributes?
  23. FareType – Spec defines this as “Indicator of which rate was charged. Options are Standard Calculated Fare (time, distance, flag drop, tolls, fees), Upfront Pricing, Flat Rate.” Maybe use trip_attributes?
  24. Tolls – Spec defines this as “Sum of any and all tolls charged for the trip, such as bridge tolls.” Maybe use trip_attributes?
  25. RateCodeID – Spec defines this as “Rate code for trip that goes 15 miles or more outside of the City & County of SF.” Maybe use trip_attributes?
  26. SFExitFee -- Spec defines this as “Fee paid by customer to exit SFO.” Maybe use trip_attributes?
  27. FlagDropAmount - Spec defines this as “Amount from the meter that results from flag drop.” Maybe use trip_attributes?
  28. OtherFees - Spec defines this as “Amount of any fees charged to the customer. Includes baggage fees. Excludes SFExitFee. “ Maybe use trip_attributes?
  29. Tip - Maybe use trip_attributes?
  30. ExtraAmount - Spec defines this as “Extra amounts charged to the customer.” Maybe use trip_attributes?
    1. PaymentType – Spec defines enumerated values: cash, credit_card, mobile, voucher, paratransit, no payment, test. Maybe use trip_attributes?
  31. TripDurationMilliseconds – Use MDS trip_duration
  32. TripDistanceMeters – Use MDS trip_distance
  33. FareTimeMilliseconds – Spec defines as “The fare time reported by the taxi meter in integer milliseconds since Unix epoch.” Maybe use trip_attributes?
  34. WaitTimeMilliseconds – Spec defines as “The wait time reported by the taxi meter in integer milliseconds since Unix epoch.” Maybe use trip_attributes?
  35. PublicationTime – Use MDS publication_time

SFMTA Telemetry API

The SFMTA Taxi Telemetry API combines the concept of a GPS breadcrumb trail and vehicle states. I think this draws on MDS Provider Trips, MDS Agency Vehicle Telemetry, and the state machine expressed in both MDS Provider Status Change and MDS Agency Vehicle Events, but interested in everyone’s thoughts on how this could be supported.

The single telemetry endpoint for both in-trip and out-of-trip telemetry makes me think of @marie-x's option 2 in the agency/provider unification.

  1. Provider_ID – same as Trips API above
  2. Taxi_Company_ID – same as Trips API above
  3. Vehicle_ID – same as Trips API above
  4. Vehicle_Placard_Number – same as Trips API above
  5. Operator_ID – same as Trips API above
  6. Driver_Status – Defined as “Indicates if this taxicab telemetry event represents the start of a driver shift, continuation of current shift, or end of shift.” Enumerated values: 1-Driver Starting Shift, 2-Driver On Shift, 3-Driver Ending Shift.
  7. Latitude – Use MDS Agency API Vehicle-Telemetry endpoint
  8. Longitude – Use MDS Agency API Vehicle-Telemetry endpoint
  9. Vehicle_Status – Enumerated values: 1-Off Duty, 2-Available, 3-Hired
  10. Event_Time_Milliseconds – could use MDS Agency Vehicle Telemetry and/or event_time from MDS Provider Status Change or MDS Agency Vehicle Events
  11. Publication_Time – same as Trips API above
schnuerle commented 2 years ago

Comparison to SFMTA Taxi Trips API

Amazing list @alexdemisch and thank you for posting! Your mapping suggestion look good, and I wonder if some of those fields are essential, e.g. maybe lat/lon is useful enough vs street address (though maybe there's a use case for this)?

Would love to see what other cities are already collecting too and incorporate what we can, so they may find it useful to adopt this in MDS 2.0. NYC TLC, Chicago, Portland, DC, and SANDAG are other agencies I know of that have a data sharing arrangement. Hearing from some of the operators in these and other areas would be helpful too.

schnuerle commented 2 years ago

Based on an OMF Passenger Services working group survey that was done in September 2020, we had over a dozen respondents share what information is already shared from taxi/TNCs. Many cities like DC, Portland, Chicago, San Diego, San Francisco, and New York City have existing data sharing agreements in place and the ability to ask for data to manage the operations of these services in the public right of way for the public good. Here are some highlights from that survey.

Kinds of data received now

Data possible in this MDS 2.0 PR:

Data not possible in this MDS 2.0 PR:

Usage of MDS instead of current method

Of the 6 cities collecting this data now, 5 of them ranked the likelihood 5 out of 5 (very likely) and one said 3 out of five (maybe) as long as using MDS was equivalent and had clear benefits.

Use Cases

Select use cases why the data the receive is needed.

  1. Demand modeling and long-term planning
  2. Infrastructure planning (e.g. pickup/dropoff zones)
  3. Revenue (e.g. fees, tolling, or congestion charging)
  4. Equitable access to services across neighborhoods and demographic groups
  5. Climate impact (e.g. GHG emissions, mode-shift, vehicle efficiency measurement)
  6. Enforcing operational rules for service providers (e.g. vehicle caps, driver registration, dispatch requirements)
  7. Driver wage fairness / minimum wage
  8. Road safety (e.g. driver hours worked, pickup/dropoff safety)
  9. Congestion management (e.g. measuring impact of passenger services, encouraging shared rides, etc.)
  10. Consumer protection (e.g. pricing fairness, efficient routing, price gouging)
  11. Enforcing operational rules for service providers (e.g. vehicle caps, driver registration, dispatch requirements)

Reasons why cities believe a standard data format is useful.

What kind of data standard would be most useful to you as it relates to passenger services?

tristan-charreton commented 2 years ago

Hello OMF working group, here are some of the questions/comments we have regarding this PR :

MulTiple trips / Journey_id Having the possibility to have multiple trips going on at the same time offers a more detailed image of the course of a taxi or TNC. Is this not redundant with the new ‘journey_id’ field ? What is the added valued of using both for the passenger services mode ? Can you provide us with a definition of a journey and a few use cases to have a deeper understanding of when a journey starts and when it ends ?

New state : stopped A new state ‘stopped’ has been introduced for trips as well as for vehicles. Is there a reason why the state stopped is suggested for passengers services? Are there other situations than the ones mentioned in the PR (pick-up / drop-off/ break) in which 'stopped' will be used ?

Roaming Is roaming not something interesting to track (e.g. to know what the traveled distance while roaming by taxis/TNCs is) ? If it is, how should roaming be identified ? Could using a different trip_type be a solution?

Reservation Something that does not seem possible for micromobility but does for passenger services : to reserve a taxi/TNC in anticipation. If it is in fact possible to do such an anticipated reservation and that this info is transmitted via MDS standards, we’ll have to track trips over a long time period, which is a problem. Plus, it raises another situation : once the reservation is made, it does not mean that the driver is heading to pick up the passenger that made the reservation and therefore it does not mean that the vehicle is on_trip. How will this case be handled ?

Edge case : enter_jurisdiction case of a taxi entering a trip while having two trips ongoing Should we have one event with two trip_id or two separate trip_enter_jurisdiction events ? (with possibly the same timestamp)

trip_attributes The trip_attributes array is not used with in passenger services. Is there a reason why this new array should be empty for passenger services ? For example, having the number of passengers for a trip could be the kind of info to have as a trip_attributes for a pooled/shared ride.

schnuerle commented 2 years ago

Hi @tristan-charreton thank you for your feedback.

MulTiple trips / Journey_id Having the possibility to have multiple trips going on at the same time offers a more detailed image of the course of a taxi or TNC. Is this not redundant with the new ‘journey_id’ field ? What is the added valued of using both for the passenger services mode ? Can you provide us with a definition of a journey and a few use cases to have a deeper understanding of when a journey starts and when it ends ?

See the diagram here for how journeys and trips can be defined for pooled trips where there is overlap. I'd also like to make this clearer in the spec, per this comment.

New state : stopped A new state ‘stopped’ has been introduced for trips as well as for vehicles. Is there a reason why the state stopped is suggested for passengers services? Are there other situations than the ones mentioned in the PR (pick-up / drop-off/ break) in which 'stopped' will be used ?

The purpose of stopped is to show where a vehicle is waiting and possibly idling in the public right of way (side street, double parking, parking spot, etc) while waiting for the next reservation or taking a break. I can also be used with trip_type to tell a story about how the vehicle is using the PROW.

Roaming Is roaming not something that interesting to track (e.g. to know what the traveled distance while roaming by taxis/TNCs is) ? If it is, how should roaming be identified ? Could using a different trip_type be a solution?

Yes I agree a roaming trip_type makes sense. I mention the idea here though I suggested awaiting_reservation as the name, which means it could be parked or moving or both. So basically defining a trip between pickup/dropoffs.

Reservation Something that does not seem possible for micromobility but does for passenger services : to reserve a taxi/TNC in anticipation. If it is in fact possible to do such an anticipated reservation and that this info is transmitted via MDS standards, we’ll have to track trips over a long time period, which is a problem. Plus, it raises another situation : once the reservation is made, it does not mean that the driver is heading to pick up the passenger that made the reservation and therefore it does not mean that the vehicle is on_trip. How will this case be handled ?

Reservations are currently allowed with micromobility. For PS it's also possible with reservation_start and reservation_end, and there are other ways to interrupt this with different cancellation types.

I don't think this is intended to be tracked when a reservation is made days or hours in advance of the trip. I think it would start to be tracked in MDS once a vehicle is being reserved or routed to a location and therefore can't be used by anyone else until the reservation is turned into an active trip. Maybe we need to clarify this in the spec?

To your last question, heading to pickup a passenger who has reserved the vehicle does start a trip, and it's on_trip, but you would know about this in more detail with the trip_type field with a reservation or pickup_request value.

Edge case : enter_jurisdiction case of a taxi entering a trip while having two trips ongoing Should we have one event with two trip_id or two separate trip_enter_jurisdiction events ? (with possibly the same timestamp)

I feel like this can be handled with journey_id tying trips together. But we should define it more here and clarify with the working group.

trip_attributes The trip_attributes array is not used with in passenger services. Is there a reason why this new array should be empty for passenger services ? For example, having the number of passengers for a trip could be the kind of info to have as a trip_attributes for a pooled/shared ride.

Yes I think it should be used, see prior comments. We need to define the values for this.

alexdemisch commented 2 years ago

Just flagging that SFMTA’s Taxi API contains two endpoints: Trips and Telemetry. It took me a bit to get a copy of the Telemetry spec, and I just updated my original post above to include that piece.

tristan-charreton commented 2 years ago

Hi @schnuerle ! Thank you for your answers !

Journey The diagram is rather clear but I am not sure I see how the journey_id helps us here avoiding overcounting MVTs. Will we expect from the operators a journey_duration and journey_distance (like we already do with the trips) ? Or is the idea to reconstruct a journey with the telemetries ? In any case, like you said, things will be easier to understand when we'll have a clear use case of a taxi's day to see how exactly data should be sent 👍

Roaming Understood, seems like a good idea. Should roaming be associated with a journey ? Or is a journey specific to a pooled ride ? These are things that will be important when we'll have to compute total MVTs.

Reservations Ok it's much simpler not to take into consideration long reservations.

to your last question, heading to pickup a passenger who has reserved the vehicle does start a trip, and it's on_trip, but you would know about this in more detail with the trip_type field with a reservation or pickup_request value.

In this PR, I see a trip_state 'reserved' so when a reservation is made, is the trip_state 'on_trip' or 'reserved' ?

Edge Cases Ok let's see how the discussions go on that matter

avatarneil commented 2 years ago

Just dropping a couple thoughts around one of the discussions happening here... Cc. @schnuerle @tristan-charreton

Roaming

I think there are two different kinds of roaming potentially being discussed here: 1) Roaming with the hope of acquiring a hail. 2) Roaming while waiting for a passenger with a reservation to arrive at the vehicle.

As far as the state machine goes, #1 would fall under the available state, and #2 would fall under the reserved state. I think that in case #2 the distance traveled should be associated with the passenger trip it is roaming for, though in #1 distance traveled should not be associated with a trip.

One thing that I've mentioned before in WG calls (and probably on Github in a few places), is I'm quite concerned about the overloading folks have been tending towards regarding Trips. I think if we define a Trip as "any time that we want to know distance", or from an MDS Provider perspective "any time we need to collect telemetry", we're setting ourselves up for having a single entity which is trying to do too much. For example, besides a passenger trip, it is completely pointless to collect some information contained within a Trip (all of the fare info). If we're really gonna go with this one-data-structure-to-rule-them-all approach, I think it might be worth considering a word other than 'Trip' for it, as to-date Trip has been 1:1 with the notion of a passenger trip.

My ideal solution, as an alternative to overloading the notion of a Trip, would be to see the addition of a Telemetry endpoint to Provider where location information can be scrapped (including for non-trip-related telemetry, optionally depending on city requirements of course), and then perhaps adding some additional properties to events, such as the distance traveled since the previous event. This would allow at an individual city-level to associate sentiment with certain sequences of events, derive analytics based on them, and not have to make any spec changes, or ask any providers to do anything different with their feeds.

jacob-sherman commented 2 years ago

Following up on Michael's request, here's a bit more information about the trip data that the city of Portland receives from the TNCs. Our regulations on data sharing are including in Portland City Code Chapter 16.40.240.J, which can be found here: https://www.portland.gov/code/16/40#toc--16-40-240-tnc-operating-responsibilities-and-prohibitions-

Here's a cut/paste of that subsection, which generally describes the type of data, as well as the mechanism and timing for data sharing:

J. Data Requirements.

  1. TNCs shall provide relevant anonymized data to the City no later than the 5 business days after the last day of the previous month pursuant to applicable data sharing agreement. Examples of relevant data may include, but are not limited to, the following:

a. Unique transaction ID number that corresponds with the passenger’s receipt;

b. Number, date, and time of fulfilled trips;

c. Trip wait time;

d. Number, date, and time of unfulfilled requests (rides the company was unable to fulfill);

e. Number, date, and time of trips declined by the driver or the company (rides declined by drivers);

f. Number of canceled rides (rides canceled by the customer);

g. Trip origin GPS, latitude and longitude; and

h. Trip destination GPS, latitude and longitude.

  1. TNCs shall submit data, pursuant to a data sharing agreement with the City and permitted companies.

  2. The data collected by the City will be, except as otherwise required by law, kept confidential by the City, used only within the City and not disclosed to third parties.

  3. In the event disclosure of such data is required by law, the City will provide TNCs notice prior to any disclosure of such data.

  4. Upon request, the TNC shall provide data identified by the Director to verify compliance with requirements pursuant to Chapter 16.40.

Note our data sharing agreements may further enumerate specifics about the data, as well as the process for data sharing. I'll see if I can connect with one of our analysts to get any additional insights on characteristics of the data.

Aside from trip data, we also have some information about drivers and their vehicles given other permit requirements. We've looked at the vehicle information recently to try to better understand fleet composition (#/% internal combustion engine, hybrid vehicle, and zero emission vehicles) which has been useful, but we also discovered that we weren't collecting that information in a standardized way (year, make, model fields exist, but not all are required and information may be mistyped, etc.) and so it's made analysis more time consuming.

nicklucius commented 2 years ago

Hi all, I'm here with Chicago's data requirements for TNCs (here we call them TNPs).

Our city council passed an ordinance that gives the department of Business Affairs and Consumer Protection the authority to require data collection as a condition to receiving a business permit. Here is the text of the ordinance.

9-115-210 Records and reports. (a) Every licensee shall keep accurate books and records of account of the licensee’s operations at the licensee’s place of business in the City for a minimum of three years. Such records shall be submitted for inspection upon the request of the Commissioner. Such records shall also be maintained in accordance with section 3-4-170 of this Code, and shall be produced in an electronic format or any other format required by the City. (b) Each licensee shall provide the following data to the Commissioner, at such times and in a format and manner prescribed by the Commissioner in rules: (1) Trip request data. A record of each request for a trip made through the licensee’s Internet-enabled application or digital platform by a potential passenger; (2) Trip data. A record of each trip which shows where a passenger is picked up and dropped off; (3) Driver data. A record of each of the licensee’s drivers who is authorized to pick up passengers using the licensee’s Internet-enabled application or digital platform; (4) Session data. A record of each driver session on the licensee’s Internet-enabled application or digital platform. For purposes of this section, a driver’s session begins when a licensee’s driver activates a mode in the licensee’s Internet-enabled application or digital platform, signaling the driver’s readiness to receive and respond to trip requests. For purposes of this section, a driver’s session ends when the driver deactivates the mode and is no longer able to receive and respond to trip requests; (5) Vehicle data. A record of each vehicle that is used by each of the licensee’s drivers for picking up passengers through the licensee’s Internet-enabled application or digital platform; (6) Location data. For every transportation network vehicle and driver combination, location snapshots captured at specified intervals for all times the driver is in session, as defined in subsection (b)(4). Each snapshot shall indicate the vehicle’s precise location and corresponding date and time; (7) Compensation data. A record of each of the licensee’s drivers who is paid an hourly rate, and any other record needed to capture actual driver pay information that is not reflected in licensee’s hourly rate compensation records; (8) Communication data. A record of each push notification or other message sent from the licensee to the licensee’s drivers or customers intended to influence the drivers’ movement or customers’ behavior, exclusive of communication data regarding a customer’s trip request and associated response; and (9) Real-time data. Only for purposes of law enforcement or emergency response, real-time tracking of the licensee’s drivers and vehicles, including access to the driver’s identifying information, GPS location data, and whether or not the driver is engaged with a passenger. If specialized hardware or software is required for real-time tracking, the licensee shall provide the specialized hardware or software to the City. (c) In addition to the requirements set forth in subsection (b) of this section and subject to subsection (d) of this section, the Commissioner may by rule require licensees to report data that the Commissioner deems to be reasonably necessary to enforce or administer this Chapter, or the ground transportation tax imposed in Chapter 3-46 of this Code. (d) This section shall not be construed to require licensees to provide personally identifiable passenger information to the Commissioner. This subsection shall be construed to allow the Commissioner to require licensees to provide GPS or any other location data regarding the physical location of the licensee’s vehicles as specified in this section or rules promulgated hereunder. (e) Each data submission to the City pursuant to this section and any rules promulgated hereunder shall be accompanied by an attestation, made under penalty of perjury, that the data submitted is accurate and complete.

With that authority, the department has passed regulations that are more specific in what data is required. The pertinent sections are as follows.

RULE TNP2.01 Data Reporting Pursuant to MCC § 9-115-210 a. A TNP licensee must submit to the City of Chicago data specified by the BACP Commissioner in a form and manner prescribed by the BACP Commissioner. b. A TNP licensee shall submit data necessary to enforce Chapter 9-115 of the MCC. Data required includes, but is not limited to, the following subjects:

  1. Trip Requests;
  2. Trips;
  3. Drivers;
  4. Sessions;
  5. Vehicles;
  6. Locations;
  7. Compensations; and
  8. Communications.

c. The data format, reporting procedure, reporting frequency, and specific elements will be prescribed by the BACP Commissioner and published at https://www.chicago.gov/tnpdatareporting. The manual located at that link will indicate the effective date of the data reporting specifications. d. A TNP licensee shall submit latitude and longitude coordinates of vehicle locations at a precision level of 4 decimal places or less; the required level of precision will be published in directives at the Website. The foregoing notwithstanding, the BACP Commissioner may request submission of latitude and longitude coordinates of vehicle locations at a precision level greater than 4 decimal places, or require the submission of location data at a frequency greater than every 60 seconds to conduct studies or analysis of traffic flow and density; public safety and infrastructure initiatives; and plans to improve transportation access to underserved passengers and neighborhoods. e. A TNP licensee must submit responsive data and information within 21 days of BACP Commissioner’s requests for data that are not published in advance at the Website. The BACP Commissioner may grant an extension for compliance with this Rule upon written request. f. Requests for information pursuant to MCC § 9-115-210(b)(9) shall be in compliance with Federal, State, and City laws including, but not limited to, the Freedom From Location Surveillance Act, codified at 725 ILCS 168/1 et seq.

And finally, the regulations incorporate the spec that is posted at chicago.github.io/tnp-reporting-manual. The spec manual gives a column-by-column data dictionary for what must be reported and how.

PlannerOnTheGo commented 2 years ago

Also to Michael's request, piping up with DC's data requirements. These are largely set by the legislation that established the data sharing requirement and added them to DC Code § 50–301.29a. General requirements for private vehicles-for-hire (PFHV is our term), available here, scroll down to 13(A): https://code.dccouncil.us/us/dc/council/code/sections/50-301.29a. Note that taxis have long had reporting included and we publish an open data version of that -

Here's a cut/paste of the PFHV data subsection:

(13)(A) Submit to the DFHV and the District Department of Transportation ("DDOT") the following information in a format approved by the Mayor, for the period July 1, 2018 through December 31, 2018 no later than February 15, 2019, and for each calendar quarter thereafter no later than 30 days after the end of that calendar quarter: (i) The total number of private vehicle-for-hire operators that utilized the digital dispatch services of the private vehicle-for-hire company in the District; (ii) A log of anonymized data relating to prearranged rides provided by private vehicle-for-hire operators that utilized the digital dispatch services of a private vehicle-for-hire company in the District that shall include the following categories of information: (I) For each trip originating and terminating inside of the District:

(III) For each trip originating inside of the District and terminating outside of the District:

(iii) The total miles driven, including both while en route to a pick-up point and while en route to a drop-off point, in the District by private vehicle-for-hire operators that utilized the digital dispatch services of the private vehicle-for-hire company in the District; (iv) The average fare and average distance for shared service trips and the average fare and average distance for private service trips; and (v) Any additional trip data that the DFHV or DDOT deems necessary for inclusion as set forth in rules adopted by the Mayor pursuant to subchapter I of Chapter 5 of Title 2; provided, that such rules shall specify that such trip data shall be anonymized and may be used only for the purposes of public safety, congestion management, and transportation planning, including curbside management, road improvements, traffic management, transit service planning, and the allocation of public monies for those purposes.

(B) The Mayor may request additional relevant information from a private vehicle-for-hire company pertaining to any trip referenced in a Metropolitan Police Department police report, provided that the report references one or more alleged criminal incidents alleged to have occurred during the time that a private vehicle-for-hire operator that utilized the digital dispatch services of the private vehicle-for-hire company was conducting a trip in the District.

(C) Any information that is received pursuant to subparagraphs (A) and (B) of this paragraph shall be deemed confidential and shall: (i) Be exempt from disclosure pursuant to § 2-532; (ii) Be safely and securely stored by the District and the District shall take all reasonable measures and efforts to protect, secure, and, when appropriate, encrypt or limit access to any data provided; and (iii) For information received pursuant to subparagraph (A), not include the personal information of passengers or private vehicle-for-hire operators that utilized the digital dispatch services of the private vehicle-for-hire company in the District.

(D) The Mayor, pursuant to subchapter I of Chapter 5 of Title 2, may issue rules to govern the sharing or publishing of conclusions and analysis derived from any information that is received pursuant to subparagraphs (A) and (B) of this paragraph; provided, that the conclusions and analysis shared shall not contain the original information that is received by the District pursuant to subparagraphs (A) and (B) of this paragraph and any shared or published data derived from the information that is received by the District pursuant to subparagraphs (A) and (B) of this paragraph shall be anonymized and aggregated across all private vehicle-for-hire companies.

(E)(i) The Mayor may enter into a confidential data sharing agreement with the Washington Metropolitan Area Transit Authority ("WMATA") or the Metropolitan Washington Council of Governments ("MWCOG") to provide those entities with anonymized and aggregated data derived from information that is received by the District pursuant to subparagraph (A) of this paragraph; provided, that the Mayor shall provide such data in a quantity and at a level of detail that is reasonably necessary for WMATA or MWCOG to conduct the analysis specified in the confidential data sharing agreement. (ii) A confidential data sharing agreement entered into pursuant to sub-subparagraph (i) of this subparagraph shall require WMATA or MWCOG to agree that: (I) The data provided shall not be disclosed by WMATA or MWCOG and shall be treated as confidential or otherwise protected for purposes of WMATA's or MWCOG's public-records requirements; (II) Notwithstanding sub-sub-subparagraph (I) of this sub-subparagraph, WMATA or MWCOG may disclose conclusions and analysis derived from the original information received pursuant subparagraph (E); provided, that the Mayor approve such disclosure and that any data disclosed by WMATA or MWCOG shall be anonymized and aggregated across all private vehicle-for-hire companies; and (III) WMATA or MWCOG shall pay the District an amount certain for each violation of the terms of the confidential data sharing agreement.

nmad-sandag commented 2 years ago

Hi all, Here is the information regarding SANDAG's (San Diego Association of Governments) data requirements/variables from household trip survey regarding TNC. Currently, SANDAG do not receive TNC data from providers. Following data is collected as a one-time thing and it was based on our TNC travel behavior survey.

Day attributes

Various attributes collected are person ID, "household_id”,“telework_time”,“num_trips_day”,“num_tnc_trips_day”, “num_shared_mobility_trips_day” etc. The number of trips (num_trips_day) is the count of total trip records associated with each person’s ID (person_id) on each travel date. Similarly, the num_tnc_trips_day variable identifies the total number of TNC records and the num_shared_mobility_trips_per variable identifies the total number of shared mobility trips associated with the person on each travel date.

Household attributes

Household attributes include Household ID Travel dates : These dates indicate the first and last days of a participant’s travel period. Home Location : “home_bg_geoid”, “home_puma”, “home_taz” etc. Reported home locations are derived from trip ends where the participant reports “Home” as the trip purpose, as well as other factors like the dwell time at those destinations. Composition: “num_workers”,“num_students” etc. Include full time workers, part time workers, and University or college students. Income characteristics : Household income was first asked via a detailed question with ten income categories plus a “prefer not to answer” option Other residential benefits and amenities data

Person attributes

Person attributes include Identifiers: Person ID and Household ID Shared mobility use (e.g., uses_carshare, carshare_freq): All adults (age 18 and older) were asked several questions about their typical transportation habits. Number of trips (num_trips_per) is the count of total trip records associated with each person’s ID (person_id) for all travel days. Similarly, the num_tnc_trips_per variable identifies the total number of TNC records associated with the person and the num_shared_mobility_trips_per variable identifies the total number of shared mobility trips associated with the person. Also, other information such as Employment characteristics, Demographic information, Work schedule and location, Transit pass availability etc. are also collected.

Trip attributes

Traveler characteristics OD characteristics : Respondents report the purpose of the trip destination in each trip survey. Departure and arrival time : Departure and arrival time indicate when a trip began and ended. Duration : Travel time is derived using the difference between the start and end timestamp of the trip. Trip leg details : Boarding/alighting location and time for each leg of travel. Dwell time : The time, in minutes, from the end of the trip until the beginning of the next trip. For trips at the end of the person’s travel period, the dwell time is calculated until 3 A.M. the following morning. Mode characteristics : Respondents could select more than one mode. The order in which respondents select modes is not recorded.

Vehicle attributes

Vehicle attributes include household ID, Vehicle number, Make, Model, Park pass, toll transponder etc. Households provided the year, make, and model of their vehicle in the recruitment survey based on a database of vehicle models from 1980-2019. Vehicle information was collected for the vehicle that the participating member indicated they use the most. Parking pass held for vehicle (for residence) and Parking pass cost for vehicle (for residence) are also collected.

schnuerle commented 2 years ago

The Massachusetts’s Dept. of Public Utilities publishes an annual report with interactive maps about TNC trips made across the state. The underlying data looks to be outlined in this Department of Public Utilities regulation 220 CMR 274.00 effective 9/22/2017 which specifies per town data on trips, drivers, financial info, data retention requirements, crashes, complaints, auditability, and more. It looks to be pretty aggregated since it's coming from a state entity, and they not as concerned with day to day operations, policy, and enforcement at the city level.

(2) Annually, a TNC shall report to the Division the following: (a) By February 1st of each calendar year, a TNC shall submit a report for the number of Rides from the previous calendar year, including:

  1. City or town where each Ride originated;
  2. City or town where each Ride ended;
  3. Aggregated and anonymized trip route and length (miles and minutes); and
  4. Location of Vehicle accidents; (b) By March 31st of each calendar year, a TNC shall report its intrastate operating revenues for the previous calendar year. If a TNC fails to report its intrastate operating revenues to the Division by March 31st of any calendar year, the Division may estimate a TNC’s intrastate operating revenues. A TNC’s intrastate operating revenue shall include but not be limited to any Rider picked up at the following:
  5. Airport;
  6. Train station;
  7. Bus terminal; or
  8. Any other kind of port

If anyone has a contact there or more information please let us know. Thanks @jacob-sherman for the info.

phillipwongg commented 2 years ago

As requested, below are the New York City Taxi and Limousine Commission’s trip data submission requirements for TNCs—what we call “high-volume for-hire services,” which currently includes Uber and Lyft.

(These requirements can also be found here: https://codelibrary.amlegal.com/codes/newyorkcity/latest/NYCrules/0-0-0-112344)

§59D-14 Operations – Trip Record Information

(a) Required Information. A High-Volume For-Hire Service must collect and transmit on a bi-weekly basis to the Commission, in a format, layout and procedure prescribed by the Commission, the following records:

(1) With respect to all trips the High-Volume For-Hire Service dispatches through a Base:

(i) The date, the time, and the location of the Passenger pickup and drop-off

(ii) The Driver’s TLC Driver License number

(iii) The dispatched Vehicle’s License number

(iv) The TLC License number of the For-Hire Base that dispatched the Vehicle

(v) The TLC License number of the For-Hire Base affiliated to the dispatched Vehicle

(vi) The total number of passengers picked up and dropped off

(vii) The total trip mileage

(viii) The date and time the Passenger requested the trip

(ix) The itemized fare for the trip including the amount of the fare, any toll, surcharge, commission rate, other deduction and any gratuity and a breakdown of the amount such passenger paid for the trip

(x) The payment the Driver received for the trip or the Driver’s hourly paid rate

(xi) If the trip enters the Congestion Zone but the pick-up did not occur in the Congestion Zone, the date, time, and location (latitude, longitude, and human-readable street address) of the point at which the vehicle entered the Congestion Zone and, if applicable, the date, time, and location (latitude, longitude, and human-readable street address) of the point at which the vehicle exited the Congestion Zone, and

(xii) An indicator as to whether the trip was administered as part of the MTA’s Access-A-Ride program.

(2) For each time a Vehicle makes itself available to be dispatched by the High-Volume For-Hire Service:

(i) The Vehicle License number

(ii) The TLC Driver License number of the Driver operating the Vehicle

(iii) The date and time at which the Vehicle became available to accept dispatches from the High-Volume For-Hire Service

(iv) The geographic position of the Vehicle during the entire time the Vehicle is available to accept dispatches from the High-Volume For-Hire Service at intervals no less frequent than every sixty (60) seconds

(v) The date and time at which the Vehicle became unavailable to accept dispatches from the High-Volume For-Hire Service

(vi) If the Vehicle enters the Congestion Zone while available to accept dispatches from the High-Volume For-Hire Service, the date, time, and location (latitude, longitude, and human-readable street address) of the point at which the Vehicle entered the Congestion Zone and, if applicable, the date, time, and location (latitude, longitude, and human-readable street address) of the point at which the Vehicle exited the Congestion Zone,

(3) The amount of time spent transporting passengers each day by each Vehicle that has made itself available to be dispatched by the High-Volume For-Hire Service, and the amount of time spent by such Vehicles between trips but not on the way to the passenger.

(4) The amount of time each Available Vehicle spends each day in the Congestion Zone, and

(5) The amount of time each Available Vehicle spends each day Cruising in the Congestion Zone.

§59D-14(a)(1)-(5) Fine: $500 for each day past the date the records are due if plead guilty before a hearing and suspension until compliance; $1,000 for each day past the date the records are due if found guilty following a hearing and suspension until compliance. Fine amount not to exceed $10,000 per bi-weekly submission of records. Appearance NOT REQUIRED

(6) Timely Submission of Trip Records.

(i) A High Volume For-Hire Service must submit trip records on a bi-weekly basis. The following penalties accrue with respect to each submission of trip records that were not submitted on time:

§59D-14(a)(6) Fine: $500 for each day past the date the records are due if plead guilty before a hearing and suspension until compliance; $1,000 for each day past the date the records are due if found guilty following a hearing and suspension until compliance. Fine amount not to exceed $10,000 per bi-weekly submission of records. Appearance NOT REQUIRED

(7) Incomplete Trip Records. With respect to all trip records submitted to TLC:

(i) Each set of submitted records must be complete and include all information listed in this subdivision and in subdivision (b) of this section. The following penalties accrue with respect to each trip for which all required information was not submitted.

§59D-14(a)(7)(i) Fine: $100 per incomplete trip record for the first ten incomplete records and suspension until compliance; $500 per each incomplete record thereafter and suspension until compliance. Fine amount not to exceed $10,000 per bi-weekly submission of records. Appearance NOT REQUIRED

(8) Inaccurate Trip Records. With respect to all trip records submitted to TLC:

(i) The records that each Base submits for any time period in which they dispatch trips must not contain inaccuracies. For example, the date, time and location of the passenger pick-up that is required by paragraph (1) of this subdivision must be accurate.

§59D-14(a)(8)(i) Fine: $100 per trip record inaccuracy for the first ten inaccuracies and suspension until compliance; $500 per inaccuracy thereafter and suspension until compliance. Fine amount not to exceed $10,000 per bi-weekly submission of records.. Appearance NOT REQUIRED

(b) Collection and Maintenance of Required Information.

(1) All records related to the location of a Vehicle, including the location at which a Vehicle enters and exits the Congestion Zone, must be collected via an in-vehicle Global Positioning System enabled device.

(2) A High-Volume For-Hire Service must ensure that all required information listed above is kept and made available for inspection by Commission representatives during regular business hours.

(3) Required trip records must be maintained by the High-Volume For-Hire Service for 18 months.

§59D-14(b) Fine: $100 if plead guilty before a hearing; $150 if found guilty following a hearing. Appearance NOT REQUIRED

(c) Special Trip Record Requirements for Minimum Driver Payments. (1) A High-Volume For-Hire Service must collect and transmit to the Commission on a bi-weekly basis, in a format, layout and procedure prescribed by the Commission, the following information for each time a Driver is available to accept dispatches from the High-Volume For-Hire Service:

(i) The Driver’s TLC Driver License number of the Driver who is available to accept dispatches from the High-Volume For-Hire Service

(ii) The Vehicle Identification Number of the Vehicle operated by the Driver specified in subparagraph (i) of this paragraph

(iii) The date and time at which the Driver became available to accept dispatches from the High-Volume For-Hire Service

(iv) The Vehicle License number of the Vehicle operated by the Driver specified in subparagraph (i) of this paragraph

(v) The geographic position of the Vehicle operated by the Driver specified in subparagraph (i) of this paragraph during the entire time the Driver is available to accept dispatches from the High-Volume For-Hire Service at an interval of no less frequent than every sixty (60) seconds

(vi) The date, time and geographic position of the Vehicle operated by the Driver specified in subparagraph (i) of this paragraph when the Driver accepts a dispatch

(vii) Total miles driven by the Driver specified in subparagraph (i) of this paragraph while the Driver was available to accept dispatches from the High-Volume For-Hire Service

(viii) Total miles driven with a Passenger while the Driver specified in subparagraph (i) of this paragraph was available to accept dispatches from the High-Volume For-Hire Service

(ix) The date and time at which the Driver specified in subparagraph (i) of this paragraph became unavailable to accept dispatches from the High-Volume For-Hire Service

(x) An indicator as to whether the Driver specified in subparagraph (i) of this paragraph or the Base made the Driver unavailable to accept dispatches from the High-Volume For-Hire Service

(xi) The total Driver earnings paid to the Driver specified in subparagraph (i) of this paragraph for the period in which the Driver was available to accept dispatches from the High-Volume For-Hire Service

(xii) The date and time at which the Driver specified in subparagraph (i) of this paragraph arrived at the pick-up location of a dispatched trip

(xiii) The date and time at which a Passenger entered the Vehicle operated by the Driver specified in subparagraph (i) of this paragraph to commence the dispatched trip

(xiv) The date and time at which a Passenger exited the Vehicle operated by the Driver specified in subparagraph (i) of this paragraph to conclude the dispatched trip

(2) A High-Volume For-Hire Service must collect and transmit to the Commission on a bi-weekly basis, in a format, layout and procedure prescribed by the Commission, for each Driver to which the High-Volume For-Hire Service dispatched a trip, a weekly statement of the Driver’s total earnings, itemized to include any deductions made from the Driver’s earnings and any payments made in addition to per-trip or hourly payments.

(3) A High-Volume For-Hire Service must collect and transmit to the Commission on a bi-weekly basis, in a format, layout and procedure prescribed by the Commission, the following additional information with respect to all dispatched calls:

(i) The itemized fare for the trip charged to the Passenger (fare, tolls, taxes, gratuity, commission rate, deductions and surcharges);

(ii) The total number of Passengers picked up and dropped off during each dispatched call referenced in paragraph (1) of subdivision (a) of this section;

(iii) The total trip mileage for each dispatched call referenced in paragraph (1) of subdivision (a) of this section;

(iv) The total trip mileage outside of the limits of the City for each dispatched call referenced in .paragraph (1) of subdivision (a) of this section;

(v) The total trip time outside of the limits of the City for each dispatched call referenced in .paragraph (1) of subdivision (a) of this section;

(vi) The date and time such trip request was made by a Passenger;

(vii) Instances where a Passenger makes multiple requests for a single, completed trip, the date and time of the latest such request;

(viii) Instances where a trip is requested but not completed because

A. The Passenger canceled the request, the Date, time and Vehicle location when the passenger canceled the request

B. The Passenger failed to show up for the requested trip, the Date and time at which the Driver canceled the request due to lack of passenger at pick-up location

C. The Driver canceled the request, the Date, time and Vehicle location when the Driver canceled the request

D. No Driver accepted the trip after the trip was requested.

(ix) The total trip time, as calculated as the time between when the Passenger entered the vehicle and when the Passenger exited the vehicle

(x) The total time between trips for the same Driver, as calculated as the time between when the prior trip ends and when the Driver receives dispatch for the subsequent trip

(xi) For trips dispatched to Drivers paid on a per-trip basis by the High-Volume For-Hire Service, the total Driver earnings paid to the Driver for each trip

(xii) For trips dispatched to Drivers paid on an hourly basis, the total Driver earnings paid to the Driver for each hour the Driver was available to receive dispatches from the High-Volume For-Hire Service.

schnuerle commented 2 years ago

Toronto is also collecting some passenger service data. Here is a link to their methodology, fields, trip records, PUDO, etc

https://www.toronto.ca/wp-content/uploads/2019/06/8f59-AppendixA_detailed_methodology_data.pdf#page=3

Table of Contents A1 Data Sources .........................................................................................A2 PTC Trip Records ...............................................................................A2 Pick-up/Drop-off Activity Data through Shared Streets ..................A2 Additional Data Provided by PTCs ....................................................A3 Transportation Tomorrow Survey (TTS)............................................A4 UTTRI Resident Survey ......................................................................A4 TTC Subway Delay Data ....................................................................A5 HERE Traffic Speed Data ..................................................................A5 Bluetooth Traffic Speed Data ...........................................................A5 A2 Methodology ..........................................................................................A5 Trip Routing........................................................................................A5 Routing Methodology before April 2017..................................A5 Routing Validation .....................................................................A7 Routing Methodology after April 2017.....................................A7 Routing Deadheading........................................................................A8 Linking Methodology .................................................................A8 Linking Optimization and Validation...................................... A10 Example Routing and Linking ........................................................ A12 Estimating Transit Alternatives to PTC Trips................................. A13 A3 Transportation Network Impacts Studies in Other Jurisdictions ..... A13

Exhibit A1-1: Current PTC Trip data provided to ML&S

  1. origin - Tagged to the nearest intersection, or municipality if outside city
  2. destination - Tagged to the nearest intersection, or municipality if outside city
  3. request time - When trip was requested (only prior to April 2017)
  4. pickup time - truncated to hour
  5. end time/duration - combined with start hour (e.g. 7:20 = start between 7am
  6. and 8 am, 20 min duration)
  7. distance in km - truncated/rounded to nearest 100 m
  8. type of service - XL, WAV, X etc.
  9. pooled trip ID - ID changes whenever vehicle is empty for pooled service
  10. trip status - Whether the trip was completed, or driver or passenger cancelled, only until April 2017
schnuerle commented 2 years ago

The Institute for Transportation Studies at University of California has done this interesting study on Sharing Mobility Data for Planning and Policy Research from 2020, which includes some charts of "Mobility Data Sharing Specifications" (not to be confused with MDS) fields and data asked for ad-hoc from 6 cities like (Table 2, page 8).

Table 3, page 11 also specifies "Sufficiency of Data Specification for Analyses" for different use cases like regulatory compliance, curb management, labor, congestion pricing, safety records, TNC vehicle electrification/EV, speed violations, transit ridership, modal comparison, etc.

https://escholarship.org/content/qt88p873g4/qt88p873g4.pdf?t=q5ng1u

schnuerle commented 1 year ago

This will be a topic at July 21's working group meeting, specifically Passenger Services Data Field Mapping, field difficulty classification, and some use case documentation.

marie-x commented 1 year ago

Working document to cogitate, coalesce and collate:

https://docs.google.com/document/d/1OV6MiH0bZSF_jswCNwObKwXO-PYdr_uWxf3no27Slk4/edit#

Reviewed at MDS WG 21 Jul 22

schnuerle commented 1 year ago

Austin, TX collects trip-level data for it's Ride Austin program including ids for passengers and drivers, start and end locations and times, etc. Details of the data and analysis of deadheading can be found in this University of Texas study and some open data here.

schnuerle commented 1 year ago

We will be having a big deep dive on the work to consolidate all of your comments, data, use cases, and feedback at this week's working group meeting. Please attend!

https://github.com/openmobilityfoundation/mobility-data-specification/wiki/Web-conference-notes,-2022.09.01-(MDS-Working-Group)

Here is the current draft of the mode in the feature branch for this PR:

https://github.com/openmobilityfoundation/mobility-data-specification/blob/feature-modes-passenger-services/modes/passenger-services.md#fare-attributes

schnuerle commented 1 year ago

I think this is at a good enough point where it can be merged back into the feature-modes branch, so that the next mode can take advantage of the work and formatting here. I'm going to do that tomorrow unless there are objections.