Closed j-d-b closed 1 year ago
I think this proposal has good intent, but is drilling too deep and not really accommodating most agency users. We need to walk before we run and in my opinion most systems and SWZ specs already require an acceptable level of accuracy for traffic sensors. If devices are not calibrated or out of service or other...they should either not be in the feed or be presented with an error or unknown condition/status. In addition, this would likely have to be manually entered by the provider or agency to populate the feed for this proposal. We believe that as much as possible needs to be automated to allow for success and widespread adoption. If traffic sensor are not providing good data 24/7, then we have a much bigger issue and should not even be providing that associated data. We either trust the date or the spec if not being satisfied and needs to be corrected ASAP. Garbage in-garbage out. Most agencies we do business with do not really care about the maintenance and calibration issues, they just want good data and system performance 24/7 and they have ways to enforce that outside of the WZDx and SWZ Device feeds. We are open to this overall discussion but need to not get caught in the weeds. We are not saying that sensors are perfect all the time, they are not. But is the sensor data representative of the current conditions to an acceptable level? If yes, then let the traffic data flow. If not than let's discuss how to address that in as automated a way as possible. The data must represent the conditions and have integrity, but not be a roadblock to the larger purpose. My two cents.
There’s also another issue that is trying to solve essentially the same problem (verify data integrity) but differently, with a timestamp per value instead of manual checkups: https://github.com/usdot-jpo-ode/wzdx/issues/253. This shows that there’s some kind of need and consumers of the data want to be confident in the data one way or another. I am guessing most data consumers just want to ensure the data is valid, but timestamps alone do not eliminate that potential of invalid data. I know internally we do many other things to ensure data integrity, but I cannot assure you that all data suppliers have the same levels of ensuring data integrity upon deployment and during the entire time in the field. If devices are calibrated and deployed correctly and accepted by an agency, why would we need to include a confidence/calibration value every minute in such a feed? I look forward to the dialogue (especially from active agencies). This may or may not be something we solve this development cycle.
@tfoster-vm , could you elaborate how it fails to accommodate agency users? Coming from that background, having standardized data formats around data calibration provides more benefits than negatives that I can think of. Having the spec marked as optional for a release or more, allows them to adjust systems to begin using before mandatory status.
The calibration itself, could yes, have to be input manually. Some devices are adding the ability to self-calibrate via implementations of ML algos and the like. Other devices without such capability, sure they'll need an interface to update.
Regardless, this doesn't stop the feed from automatically updating. Additionally, this gives manufacturers the ability to add business rules such as if device moved more than threshold distance -> mark for calibration. As @rdsheckler mentioned in https://github.com/usdot-jpo-ode/wzdx/discussions/262#discussioncomment-2382057_ this is something others are looking to do regardless. The agencies and researchers I've worked with have always mentioned that more filtering ability is a positive. This gives them an additional criteria for filtering with more granularity than Source->Score applied to all in feed.
Additionally, by the device in the field providing a score that can be used at ingestion time, it provides the OEM the ability to help the Agency develop rules for automatically flagging devices for calibration. We've all been a part of a workzone where a device may be deployed initially and calibrated, then it's moved/turned by a worker for whatever reason. Maybe it gets put back correctly, maybe not. This allows business rules to be developed to help flag this so the agency can charge the device maintainer, or agency staff to return for calibration.
Depending on the TMS exporting the data, they may choose to include all devices in the system. So you end up with both WZ specific devices, as well all permanent ITS devices that exist within a given WZ, or a mixture of both. Ideally the system should be able to track the reliability/calibration of the data. They may also choose to pass through directly the data as presented from the device, or apply additional business rules prior to publishing and giving their mark of approval. This can help reduce the need for Agency staff to login and toggle specific devices as publish/don't publish by generating additional business rules around publish scores.
@benafischer94 in general I think the more manual entry there is the less likely it will be utilized. I think most of what you are proposing is manual entry. If I am wrong, please let me know. I suggest we see if there is wider support for this, since the only comments are from the two of us. I will solicit comments at our next subgroup meeting and you can feel to add context or specifics during that meeting.
Closed based on SWZ co-chairs discussion on 2022-09-09:
Discussed in https://github.com/usdot-jpo-ode/wzdx/discussions/262