Onboard Scoobi to MDS

[johnclary]

Onboard provider Scoobi to MDS.

We've been provided with API access and are currently testing the production API based on trips completed on 9/23.

[johnclary]

Email to Steve after testing:

We've attempted to pull trips from the test rides, but the API is returning the same set of ~30 trips regardless of the end_time we use to query.

All of the trips returned are timestamped on Sep 27. As well, the lon/lat coordinate pairs are reversed, and upon reversing them it's clear these trips took place in Pittsburgh.

Please advise.

And response:

I published it and flipped the order of the lat/long in the array; it should be good now.

Could you give me an example of what your passing in for the date time val? I’m able to specify different times and it converts ok.

I replied to Steve with an example query URL we're using.

[johnclary]

I've been using this script to convert the responses sergio provided into geojson that can be inspected on geojson.io

"""Creates a geojson feature collection from MDS trip. Inspect by loading in geojson.io"""
from datetime import datetime
import os
import json
import requests

TOKEN = os.getenv("TOKEN")
ENDPOINT = os.getenv("ENDPOINT")
END_TIME = "2021-09-23T18"

def get_data():
  headers = {"Authorization": f"Bearer {TOKEN}"}
  params = {"end_time": END_TIME}
  res = requests.get(ENDPOINT, headers=headers, params=params)
  res.raise_for_status()
  return res.json()["data"]["trips"]

data = get_data()

trips = {'type': 'FeatureCollection', 'features':[]}

for trip in data:
  feature = {'type': 'Feature', 'properties': {}, 'geometry': {'type': 'MultiPoint', 'coordinates': []}}
  print(str(datetime.fromtimestamp(trip["end_time"]/1000)))
  waypoints = trip["route"]["features"]
  if waypoints[0]["geometry"]["coordinates"][1] > 40:
    continue

  for w in waypoints:
    feature["geometry"]["coordinates"].append(w["geometry"]["coordinates"])

  # drop the route and assign remaning properties to feature.properties
  trip.pop("route")
  for key, val in trip.items():
    if "time" in key:
      # humanize timestamps
      val = str(datetime.fromtimestamp(val/1000))
    feature["properties"][key] = val
  trips["features"].append(feature)

with open(f"{END_TIME}.geo.json", "w") as fout:
  fout.write(json.dumps(trips))

[johnclary]

I notified Steve of these issues:

We're able to query for trips and the lat/lon are now correct. However, our queries return trips in the future as well as trips that are in Pittsburgh.

As well, the sole Austin trip has three identical coordinate pairs. We would expect these to be different (assuming the vehicle moved during the trip).

Here's the query: /provider/dockless/trips?end_time=2021-09-23T17

We need:

• Only trips in Austin
• The query to return trips that match the requested end_time

[sergiogcx]

Thanks for the code @johnclary ! It's pretty interesting seeing readable timestamps as part of the properties each feature. I have been doing it the lazy-man's way with Smart JSON Editor, Ctrl+C (for copy) than paste directly into geojson.io, the disadvantage being that you can only copy-paste one trip at the time:

[johnclary]

whoa, cool editor though!

[johnclary]

updated the python script ☝️ to fetch from the provider endpoint and attach all trip properties to the geojson feature

[johnclary]

TLDR: Scoobi meets our needs but is still working to meet RideReport's requirements.

Thanks, Steve. I reviewed the Pittsburgh trips. The route features appear to be timestamped with the time of the API request. They are often identical and do not fall within the range of the start/end times. As well, the number of route points is quite low relative to the duration of the trip.

Our internal reporting relies on only the beginning and end of the route and does not reference the route feature timestamps. From our side your spec meets our needs, assuming the starting/ending route features will be accurate (ie, assuming that the issue that occurred with the Austin test trip has been resolved).

You will need to continue to work with RideReport to refine your spec as they rely on a more robust feature set.

[amenity]

@sergiogcx - any updates on this?

[sergiogcx]

@amenity @patrickm02L I do not have any updates, last time I tried scoobi's API with @johnclary it was a bit of a mess.

While the API returned the right format of data, it had a few problems:

1. Their API always returned 3 trips
2. Their test coordinates were wrong, they where in Antartica.
3. We had some issues with their timestamps not being real. The trips were just made up and you could tell because when we ran the API for the same time block the timestamps were always different, even for the same trips.

This was a while back, I wonder if their API has been corrected since, but I have not kept track so I defer to you guys.

[sergiogcx]

Moved to blocked because I do not know the status, maybe that's the next thing to do. Maybe I need to retry again.

[johnclary]

Andrea tells us that Scoobi will go-live on 11/1. I'll send a calendar reminder for us to turn on their DAG

[patrickm02L]

@johnclary Sergio walked me through turning on the DAG for Scoobi. It's live and we're monitoring it in advance of go-live on 11/1.

[johnclary]

Excellent—thanks!

[patrickm02L]

Refer to Shared Mobility dashboard to see trips. Everything else looks good.