Street QUality IDentification ("SQUID") leveraging Open Street Cam ("OSC")

[patwater]

Street QUality IDentification ("SQUID") is an open source data pipeline and analytics framework for assessing street quality.

This approach utilizes the open source Open Street Cam to collect GPS, accelerometer and imagery data to be able to literally see the Ground Truth.

This Public CEO article describes the improvements over current approaches to measure the Pavement Condition Index ("PCI") through a case study in Los Angeles.

Here are a set of example dashboards for select cities:

• https://demo.streetsdatacollaborative.org/commute/ (Example dashboard described in public CEO article before the bike lane was fixed)
• https://demo.streetsdatacollaborative.org/commute_fixed/ (Same commute above after being fixed by the LA Bureau of Street Services -- 😄 )
• https://demo.streetsdatacollaborative.org/San-Francisco/
• https://demo.streetsdatacollaborative.org/Berkeley/
• https://demo.streetsdatacollaborative.org/San-Diego/
• https://demo.streetsdatacollaborative.org/Oakland/

Here is a blog post describing how this sort of prototype analytics can be easily be spun up utilizing existing OSC data.

Here is a blog post providing documentation on how to collect new OSC data (bike specific though ultimately SQUID is mode agnostic).

Note this integrates as part of a larger vision for inter-city collaboration to share data and work together to navigate the seismic shifts underway in urban mobility. See StreetsDataCollaborative.org for details.

That tooling is available here on GitHub: https://github.com/streets-data-collaborative

Thank you for taking the time to review and please don't hesitate to reach out if you have any questions!

CC @vr00n @schnuerle @thenameisdave

[patwater]

@schnuerle any questions or issues we can address? Happy to hop on a call when convenient to discuss. To support this inter-governmental urban mobility / streets data collaboration we're scoping a Brigade Action Team focused on bikes with folks in Open Oakland, Hack for LA, MapTime etc. See here: https://docs.google.com/document/d/1TCpl9B_NZJIVSODkH5a6TJcPbccqj3JeitJS5phh_js/edit#

[schnuerle]

I think it sounds like a great project. Can you flesh it out a bit, like with these kinds of details from the Waze project?

• Potential Sponsors: Amazon AWS, Slingshot
• Potential Promoters: Waze
• Potential Future Collaborators: Microsoft Azure, Google Cloud, Code for America
• Possible Constraints: Must be a Waze CCP partner (signed legal agreement with Waze)

Does it have its own Git repo yet, with most of the info you provided in the issue description above, plus things like a roadmap, defined issues, wiki, project plan, etc in Git to help organize the collaboration?

If you do some of that I can add it to the OGC home page as a project, which can get some more support for it.

[patwater]

Yes there are several SQUID repos for pieces of the data integration and a MVP data analytics tool available here: https://github.com/streets-data-collaborative

The technical context as part of an integrated vision for supporting cities in tackling common urban mobility challenges is articulated here: http://streetsdatacollaborative.org/

They key need in terms of collaboration are:

1. OSC data collection either on city vehicles or bike volunteers (this is they key constraint as the data needs to be collected)
2. Public works technical leaders inside and outside of cities to collaboratively work on modernizing the Pavement Condition Index = PCI (existing data standard for street quality)

We have some potential sponsors though still in discussions so happy to provide those in a non-public forum at this stage.

Potential future collaborators include the CfA Bike BAT for data collection, academic institutions for collaboration on PCI modernization (civil engineering) and computer vision classifications (CS), Open AI for collaboration on computer vision, as well as new and legacy car companies.

[patwater]

@schnuerle are you going to be at CfA brigade summit? Would be great to catch up tomorrow if so!

[schnuerle]

I couldn't make the summit this year, but heard @govintheopen was talked about a bit. https://twitter.com/brendanbabb/status/1003492985449803776

I added SQUID to the https://www.govintheopen.com/ home page under proposed projects and filled out what I could. Feel free to leave edits here or do a PR to make changes and keep it up to date. https://github.com/GovInTheOpen/GitBook-Documents

Looks like that puts us at steps 5 and 6 of the ingestion process. https://www.govintheopen.com/how-to-run-an-ogc-project.html

Besides looking for organizations to fund, can you flesh out the how it can be easily deployed by a gov, make sure it is well documented including some use cases, and ideas on how it can be deployed to cloud accounts easily (eg, Terraform, CloudFormation, Docker, etc).

[mvexel]

👋 Just stumbled upon this. I'm on the team that builds and runs OpenStreetCam. Let me know if you need any background / info from our side. We're not affiliated with SQUID but are aware of this project.

[patwater]

We saw the new OSC API -- very exciting! https://apollo-api-v2.openstreetcam.org/apolloService/

Would love to see more detailed documentation of that API as would be hugely helpful for the SQUID project. Thanks much!

[aplannersguide]

@patwater I'm intrigued by this project and know all the players here in Chattanooga needed to make it happen. Ideally I would like to pilot test on a single garbage truck and then roll out to the rest of the fleet. So if this project keeps moving forward I would like to be included. I'm tapped out as far as capacity goes so it would depend if I could find other willing partners to lead the pilot.

[vr00n]

@aplannersguide Hi there Tim. Sounds great. I can help with the deployment but I am limited to working on evenings and weekends. Perhaps we can arrange for a call next week to better understand scope and expectations? I can be reached at varun@argolabs.org

That said, know that garbage trucks are not ideal as the accelerometer data that comes off them can get quite noisy but if you believe that they are the best way to instrument a fleet that drives most of Chattanooga's network then by all means

1. try and mount any smartphone with decent storage space (SD card) on the behind of that truck.
2. find a way to power it (you will likely need a very long usb cable or mount the phone along with a USB charging device).
3. Run the Open Street Cam app
4. Upload the images using Wifi

When working with the city of Syracuse with the Raspberry Pi back in 2016 - here's how the mounting looked. Lots of duck tape was involved :) We collected close to 500 miles (100,000 images) of street conditions on this setup.

More context and story here: https://hackster.io/argo/squid-street-quality-identification-device-a43367tph

[aplannersguide]

Thanks @vr00n! I was thinking about the noise problem when using garbage trucks just this morning and was wondering if there could be a way to post process the accelerometer data or the SQUID score. The main reason I was thinking garbage was because that fleet gives us almost full coverage of our city's streets on a regular basis. I'll have to ask around to see if other depts would be better candidates (police, code enforcement, land development office).

Thanks for the suggestions on the setup. I wasn't thinking of space or power yet but that makes total sense. Is OSC a hog on both accounts? I was thinking tie straps and duck tape make the most sense for the prototyping phase. I'm a big fan of rapid prototyping and quick learning over getting it perfect.

[vr00n]

@aplannersguide

re: post processing the accelerometer data or the SQUID score.

We created https://github.com/Streets-Data-Collaborative/osc-tools for this but it appears that OSC has updated their data shapes and may even release a more formalized API so its a bit outdated unfortuanately.

If the data collection uses the same type of phone throughout, we can work together to parse the data.

Code enforcement could be an interesting fleet to instrument. I would also ask you to get on a bike and survey the bike lanes in your neighborhood to get familiar with the entire data collection process.

Happy to continue the conversation on this thread.

See here for my conversation with OSC devs https://github.com/openstreetcam/openstreetcam.org/issues/109