ross-spencer
commented
1 year ago SPARQL query for TrID:
SELECT ?uriLabel ?uri ?puid ?signature ?sourceLabel ?relativityLabel WHERE {
SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en". }
?uri wdt:P31/wdt:P279* wd:Q235557.
?uri p:P4152 ?statement.
optional { ?uri wdt:P2748 ?puid. }
?statement ps:P4152 ?signature.
?statement prov:wasDerivedFrom ?reference.
optional { ?statement pq:P2210 ?relativity . }
?reference pr:P248 wd:Q41799265.
?reference pr:P248 ?source.
}Results: 2023-02-02-trid-query.zip
thorsted
Description of problem
TrID signatures are of a much less quality than something like PRONOM, largely because of the learning algorithm it uses. Partially too because TrID's confidence ratings haven't been translated into Wikidata.
An initial analysis shows a large number of TrID signatures are below 10 bytes in length with over 900 <= 3. All sequences are BOF sequences, and there are approximately ~1500 duplicates.
So, we need to take a look at these signatures and understand how bad the problem is, and what can be done about it.
Permalink
A good place for discussion, a la the new XML discussion: https://www.wikidata.org/w/index.php?title=Talk:Q41799265&action=edit&redlink=1
Analysis spreadsheet: https://docs.google.com/spreadsheets/d/1M_7K8RpHI2UM2OepUWvNRA4u8KbuEA_o_UH_lO6_4eA/edit?usp=sharing
Related to: https://docs.google.com/document/d/1jMXcbFHVtw8mdNNJNZED7DweXhDvOQ5K2SW_ZVfNJi0/