v1 mds derived metrics

[kpwebb]

@morganherlocker here's a preliminary list of metrics that can be derived from MDS:

Zonal (H3 cell):

• Availability (weighted average fractional vehicles per hour)
• Uniques (vehicles per hour)
• Activity (trips per hour - can be further divided into things like "average trip length/duration/etc." )
• Utilization (% of available capacity used per hour)
• Zone to Zone flows (trips per hour)

Street-linked (ShSt references):

• Unlinked trip start/end (trip per hour per ShSt bin)
• Segment volume (trips per hour traversing ShSt segment)
• Segment flows (trips across segments measure as % of trips traversing that traversed ShSt segment A that also traversed segment B)

let's break these into metric specific tickets so we can keep track of the discussion on implementation details.

[morganherlocker]

@kpwebb I took your metrics list and attempted to normalize it into a set of key features per metric for the official specification. Outstanding questions below.

Metrics Specification

• Vehicles
  • description: count of unique vehicles
  • spatial: global, h3, shst
  • temporal: 1h
  • filter: >=2
  • source: /trips
• Pickups
  • description: count of unique trips
  • spatial: global, h3, shst
  • temporal: 1h
  • filter: >=2
  • source: /trips
• Dropoffs
  • description: count of unique trips
  • spatial: global, h3, shst
  • temporal: 1h
  • filter: >=2
  • source: /trips
• Pickups Via
  • description: per zone count of trips originating in target zone
  • spatial: h3+h3
  • temporal: 1h
  • filter: >=5
  • source: /trips
• Dropoffs Via
  • description: per zone count of trips ending in target zone
  • spatial: h3+h3
  • temporal: 1h
  • filter: >=5
  • source: /trips
• Utilization
  • description: aggregate % of time any vehicles in zone were in use
  • spatial: global, h3
  • temporal: live
  • filter: N/A
  • source: /status_changes
• Availability
  • description: aggregate % of time any vehicles in zone were not in use
  • spatial: global, h3
  • temporal: live
  • filter: N/A
  • source: /status_changes

Questions

• Availability/Utilization both require a definition of "in zone", since a vehicle would normally cross many zones without registering a status event. One way to do this would be to assign every vehicle a single zone at all times of day, with the zone changing only at time of dropoff. We should poke holes in this to make sure the metrics would not be statistically meaningless.
• I added a spatial aggregation called "global", but had not heard this discussed before. Is a city-wide aggregation useful for the metrics I have it labeled for?
• I was liberal in my interpretation of "X per hour" where referenced. "X per hour" is an instantaneous rolling metric, where as the metrics I described are more like "in hour X". These metrics would be updated at the top of the hour and would not change depending on when a query was run within the hour. Either aggregation strategy is feasible, but "in hour X" requires less storage and compute, and the two are mostly functionally equivalent.
• OD segment flows per pair may not be viable with the volumes of data a micromobility service would be generating, which may mean not enough useful data being reported. As an illustration, imagine a small city with about 50k street edges, such as Honolulu. If we had medium-low resolution binning with ~100 zones, we would split our dimensions 10,000 dimensions = 100 zones * 100 zones we would already have significant fragmentation of aggregates, made impossibly small when including roads 500,000,000 dimensions = 10,000 OD combinations * 50,000 street edges. With that much fragmentation, edge volume mode would be 0, with just a few lighting up with volume of 1, essentially reconstructing a raw GPS map. There will be some amount of concentration on OD pairs and edges, but I think we would still need millions of trips within the query window to show any useful patterns.

[morganherlocker]

Update: I have edited the specification draft to include the MDS provider endpoint source for each metric.

[morganherlocker]

These have all shipped in one form or another, and are documented on the readme. There is work in progress on utilization & availability metrics as I am rewriting those today based on user feedback. I will track additional work in smaller issues.