joecastiglione
commented
1 year ago "ODOT: When ODOT took this on for the Oregon ABM, the intial value was a visualizer (HTML) that did all the comparison of the model to the calibration target data. The idea was that the target data was processed to look just like ABM output, then the visualizer was pointed to both the target dataset and the ABM dataset to do the comparison. This allowed ODOT to have the full set of calibration comparisons instantly at the end of each ABM run. Then after the calibration was completed, ODOT had a visualizer that allowed each future scenario to be compared to other reference ABM runs to quickly understand how the scenarios were different. I recommend that this be approach that way - as an automated calibration visualizer that is than directly usable as a scenario to scenario visualizer.
SANDAG: I think this is a very useful feature. A few of the visualization, QAQC, and data modeling items on the list can be comined to form a common data modeling/visulization/reporting/QAQC platform.
MTC: Sounds interesting but similar to 21, interested in examples, technology options."
Several ad hoc reporting, validation, and testing scripts in Python and R have been developed to validate, verify, and debug ActivitySim model results. It would be good to build an ActivitySim QA/QC/reporting framework that is included with the software and can be used by users and developers. This would likely take the form of Python software with support for creating data summaries, maps, charts, etc. as well. With a framework in place, the model builder could implement several QA checks to ensure the model is behaving as expected. As the ActivitySim platform continues to mature and the user base grows, features like this will likely become increasingly important.