MarcoMinghini
commented
7 years ago Very nice observation. Time dimension is crucial for OSM as - in contrast to any other authoritative database - it evolves dinamically and changes at any time. This is a useful addition we can make to our tool, however I can say that that our procedure was conceived with a different purpose: compare two datasets. Then it is only up to the user to know which datasets to compare and know their characteristics. So the user can use an OSM dataset dating back to a specific date. But yes, we may also think to allow users to extract OSM data for any specific date. Definitely an intrinsic quality evaluation (i.e. evaluation of OSM quality using only OSM) would be great, perhaps compared to the extrinsic one we are using. There is already a lot of literature about that!
I'm thinking about using this toolkit. However, it has the same limitation which has bothered me in all similar analysis: it is a snapshot affair. In the case of REF data, we're looking at a snapshot of a finished product, of which we can assume the quality will stay more or less the same over time (with a tendency to stay the same or worsen). In the case of OSM, there is a very obvious tendency to improve over time; especially in the time frame 2007-2012. An example analysis of this evolution for Flanders OSM data available here.
An analysis taking into account the time dimension of OSM data, could conceivable indicate when the OSM dataset will (or has is in the past) achieved or surpassed the quality of the REF dataset. Using the OSM-history-importer (unfortunately unmaintained, and with a known bug regarding deletions), it is relatively straightforward to generate a shapefile of the state of the road network on any given date.
In a later phase, it might be necessary to take into account different time slices of the REF dataset too.
A further research question that could be answered with this approach, is to what degree OSM road networks can be evaluated without the need of an external dataset.