Analyse count discrepancies across KB, PDB and PDB2

VirtualFlyBrain / vfb-pipeline-config

Pipeline 2.0 Configuration

Apache License 2.0

0 stars 1 forks source link

Analyse count discrepancies across KB, PDB and PDB2 #21

Closed matentzn closed 4 years ago

matentzn commented 4 years ago

Number of individuals between ts and pdb seems fine.
Number of object properties a bit odd: considerably more in PDB than TS%20%0AWHERE%20%7B%0A%0A%09%3Fa%20a%20owl%3AObjectProperty%0A%7D&limit_query=100&infer=true&). This could be possible because of a quirk in RDF however that it does not understand if something is a property, but the OWLAPI understand this from the context.
Huge difference in number of classes, so this is where I will start looking probably unsats

matentzn commented 4 years ago

Ok @dosumis @Robbie1977 I need some help with this:

Class counts:

PDB1: 74955
PDB2: 43803
PDB-ALPHA: 74959
KB1: 42538
KB2: 42488
TS: 50232

Any ideas where I should be looking? Is it possible that the 50K classes in the triple store is correct, given that the slicing in the collect data pipeline may be more rigorous now the way we do it? Same with individual counts

PDB1: 269246
PDB2: 143411
PDB-ALPHA: 291912
KB1: 218674
KB2: 170293
TS: 143410

dosumis commented 4 years ago

I wouldn't worry about comparing class numbers

matentzn commented 4 years ago

At least I worry TS->PDB2

matentzn commented 4 years ago

TS discrepancy was mostly because the counting query took into account anonymous expression... (blank nodes)

matentzn commented 4 years ago

Ok we will close this, as most of the discrepancies have been accounted for. Some of the larger ones are due to Flybase side loading - expression patterns and the likes.