search

agroportal / project-management

Repository used to consolidate documentation about the AgroPortal project and track content related issues.
http://agroportal.lirmm.fr
7 stars 0 forks source link

0ntologies diagnostics (January 2022) - Ontologies with RDF data, without SOLR data #198

Closed syphax-bouazzouni closed 2 years ago

syphax-bouazzouni commented 2 years ago

Ontologies with RDF data, without SOLR data:


http://bioportal.bioontology.org/ontologies/ATOL
        flat=
        submissionId=154
        hasOntologyLanguage=OWL
        metrics=averageChildCount:3,classes:2352,classesWithMoreThan25Children:2,classesWithNoDefinition:2352,classesWithOneChild:136,individuals:0,maxChildCount:35,maxDepth:12,properties:0
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ANNOTATOR,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/CO_320
        flat=
        submissionId=63
        hasOntologyLanguage=OWL
        metrics=averageChildCount:10,classes:2558,classesWithMoreThan25Children:10,classesWithNoDefinition:1935,classesWithOneChild:1,individuals:0,maxChildCount:399,maxDepth:3,properties:3
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ANNOTATOR,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/CO_321
        flat=
        submissionId=63
        hasOntologyLanguage=OWL
        metrics=averageChildCount:22,classes:1821,classesWithMoreThan25Children:13,classesWithNoDefinition:1135,classesWithOneChild:2,individuals:0,maxChildCount:465,maxDepth:3,properties:3
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ANNOTATOR,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/CO_322
        flat=
        submissionId=62
        hasOntologyLanguage=OWL
        metrics=averageChildCount:18,classes:1078,classesWithMoreThan25Children:10,classesWithNoDefinition:630,classesWithOneChild:3,individuals:0,maxChildCount:344,maxDepth:3,properties:3
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ANNOTATOR,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/CO_324
        flat=
        submissionId=60
        hasOntologyLanguage=OWL
        metrics=averageChildCount:16,classes:1133,classesWithMoreThan25Children:9,classesWithNoDefinition:657,classesWithOneChild:1,individuals:0,maxChildCount:324,maxDepth:3,properties:3
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ANNOTATOR,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/CO_325
        flat=
        submissionId=61
        hasOntologyLanguage=OWL
        metrics=averageChildCount:14,classes:1427,classesWithMoreThan25Children:10,classesWithNoDefinition:832,classesWithOneChild:3,individuals:0,maxChildCount:355,maxDepth:3,properties:3
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ANNOTATOR,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/CO_327
        flat=
        submissionId=60
        hasOntologyLanguage=OWL
        metrics=averageChildCount:8,classes:238,classesWithMoreThan25Children:2,classesWithNoDefinition:154,classesWithOneChild:1,individuals:0,maxChildCount:43,maxDepth:3,properties:3
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ANNOTATOR,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/CO_330
        flat=
        submissionId=57
        hasOntologyLanguage=OWL
        metrics=averageChildCount:10,classes:1480,classesWithMoreThan25Children:10,classesWithNoDefinition:1043,classesWithOneChild:0,individuals:0,maxChildCount:261,maxDepth:3,properties:3
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ANNOTATOR,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/CO_331
        flat=
        submissionId=61
        hasOntologyLanguage=OWL
        metrics=averageChildCount:10,classes:1921,classesWithMoreThan25Children:9,classesWithNoDefinition:1361,classesWithOneChild:0,individuals:0,maxChildCount:307,maxDepth:3,properties:3
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ANNOTATOR,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/CO_334
        flat=
        submissionId=60
        hasOntologyLanguage=OWL
        metrics=averageChildCount:10,classes:1566,classesWithMoreThan25Children:12,classesWithNoDefinition:1122,classesWithOneChild:1,individuals:0,maxChildCount:265,maxDepth:3,properties:3
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ANNOTATOR,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/CO_337
        flat=
        submissionId=63
        hasOntologyLanguage=OWL
        metrics=averageChildCount:11,classes:765,classesWithMoreThan25Children:8,classesWithNoDefinition:503,classesWithOneChild:2,individuals:0,maxChildCount:164,maxDepth:3,properties:3
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ANNOTATOR,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/CO_338
        flat=
        submissionId=63
        hasOntologyLanguage=OWL
        metrics=averageChildCount:10,classes:440,classesWithMoreThan25Children:4,classesWithNoDefinition:282,classesWithOneChild:0,individuals:0,maxChildCount:80,maxDepth:3,properties:3
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ANNOTATOR,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/CO_339
        flat=
        submissionId=63
        hasOntologyLanguage=OWL
        metrics=averageChildCount:9,classes:403,classesWithMoreThan25Children:3,classesWithNoDefinition:273,classesWithOneChild:1,individuals:0,maxChildCount:66,maxDepth:3,properties:3
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ANNOTATOR,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/CO_340
        flat=
        submissionId=60
        hasOntologyLanguage=OWL
        metrics=averageChildCount:11,classes:1078,classesWithMoreThan25Children:10,classesWithNoDefinition:759,classesWithOneChild:0,individuals:0,maxChildCount:173,maxDepth:3,properties:3
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ANNOTATOR,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/CO_341
        flat=
        submissionId=62
        hasOntologyLanguage=OWL
        metrics=averageChildCount:9,classes:321,classesWithMoreThan25Children:2,classesWithNoDefinition:196,classesWithOneChild:1,individuals:0,maxChildCount:68,maxDepth:3,properties:3
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ANNOTATOR,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/CO_343
        flat=
        submissionId=56
        hasOntologyLanguage=OWL
        metrics=averageChildCount:12,classes:810,classesWithMoreThan25Children:7,classesWithNoDefinition:431,classesWithOneChild:3,individuals:0,maxChildCount:206,maxDepth:3,properties:3
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ANNOTATOR,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/CO_345
        flat=
        submissionId=57
        hasOntologyLanguage=OWL
        metrics=METRICS_MISSING
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_MISSING_STATUS,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/CO_346
        flat=
        submissionId=59
        hasOntologyLanguage=OWL
        metrics=averageChildCount:8,classes:461,classesWithMoreThan25Children:3,classesWithNoDefinition:310,classesWithOneChild:3,individuals:0,maxChildCount:89,maxDepth:3,properties:3
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ANNOTATOR,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/CO_348
        flat=
        submissionId=58
        hasOntologyLanguage=OWL
        metrics=averageChildCount:17,classes:480,classesWithMoreThan25Children:4,classesWithNoDefinition:251,classesWithOneChild:3,individuals:0,maxChildCount:151,maxDepth:3,properties:3
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ANNOTATOR,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/CO_350
        flat=
        submissionId=57
        hasOntologyLanguage=OWL
        metrics=averageChildCount:30,classes:997,classesWithMoreThan25Children:9,classesWithNoDefinition:460,classesWithOneChild:0,individuals:0,maxChildCount:328,maxDepth:3,properties:3
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ANNOTATOR,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/CO_358
        flat=
        submissionId=53
        hasOntologyLanguage=OWL
        metrics=averageChildCount:15,classes:1100,classesWithMoreThan25Children:10,classesWithNoDefinition:650,classesWithOneChild:1,individuals:0,maxChildCount:279,maxDepth:3,properties:3
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ANNOTATOR,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/CO_360
        flat=
        submissionId=50
        hasOntologyLanguage=OWL
        metrics=averageChildCount:10,classes:206,classesWithMoreThan25Children:3,classesWithNoDefinition:102,classesWithOneChild:4,individuals:0,maxChildCount:71,maxDepth:3,properties:3
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ANNOTATOR,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/CO_365
        flat=
        submissionId=51
        hasOntologyLanguage=OWL
        metrics=averageChildCount:11,classes:443,classesWithMoreThan25Children:3,classesWithNoDefinition:268,classesWithOneChild:2,individuals:0,maxChildCount:94,maxDepth:3,properties:3
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_MISSING_STATUS,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/DEMETER-AIM
        flat=
        submissionId=5
        hasOntologyLanguage=OWL
        metrics=averageChildCount:7,classes:180,classesWithMoreThan25Children:3,classesWithNoDefinition:87,classesWithOneChild:14,individuals:137,maxChildCount:45,maxDepth:6,properties:286
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ERROR_DIFF,ANNOTATOR,INDEXED,INDEX_ERROR:submissionId:4,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/GAO
        flat=
        submissionId=4
        hasOntologyLanguage=OWL
        metrics=averageChildCount:5,classes:65,classesWithMoreThan25Children:0,classesWithNoDefinition:48,classesWithOneChild:0,individuals:0,maxChildCount:5,maxDepth:1,properties:0
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ANNOTATOR,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_AVAILABLE
jonquet commented 2 years ago

Friday Fev 4th : bin/ncbo_ontology_process -o THVIANDE,THESAE,GECO,GAO,DSW,CO_360,CO_358, CO_350 => It seems a bug has generated "ERROR_RDF" issues for almost all these ontologies (except THVIANDE) (maybe the issue was cause by not running the script with ontoportal user)

Monday Fev 7th : bin/ncbo_ontology_process -o THESAE bin/ncbo_ontology_process -o GECO,GAO,DSW,CO_360,CO_358,CO_350,CO_324,CO_327,CO_330,CO_337,CO_338,CO_339,CO_340 => Last command failed after CO_358

jonquet commented 2 years ago

List of ontologies to reprocess

(while monitoring the server)

jonquet commented 2 years ago

Feb 11th: bin/ncbo_ontology_process -o CO_324,CO_327,CO_330,CO_337,CO_338,CO_339

syphax-bouazzouni commented 2 years ago

Feb 14th:

jonquet commented 2 years ago

Feb 14th: bin/ncbo_ontology_process -o CO_341,CO_343,CO_346,CO_348,CO_350

syphax-bouazzouni commented 2 years ago

New ones from Admin UI:

jonquet commented 2 years ago

I confirm that parsing GECO in StagePortal fix the indexing issue.

image

However the problem still appear for the Annotator that certainly adopts the same behaviour (ignoring labels in French) when building its dictionary independently of the language parameters of the portal:

image

However, for the Annotator, this is a desired behaviour. With AgroPortal not fully multulingual, we have defined the Annotator to handle only labels in english or without language tags.

jonquet commented 2 years ago

Process to resolve parsing issues for up-listed ontologies with mostly French labels:

jonquet commented 2 years ago

After discussion we decided that we will not change AgroPorta settings to avoid creating a special case with French language. See #219 for details.

Fixing the the up-listed issues in admin requires to update the indexation test that is not in synch with what's actually implemented. @syphax-bouazzouni to look into this.

To avoid having multilingual labels included in the search index by mistake we will be fixing ontologies => #221

syphax-bouazzouni commented 2 years ago

Our problems with indexing were related to our handling of multilingual values, where for the case of Agroportal we take only attributes values that are tagged with the English language or that have no language tag.

So for the case of resources (ontologies) that are only in French (or any other language), we don't get any values from the triple store.

Our solution was to update this behavior like detailed here: https://github.com/ontoportal-lirmm/goo/issues/6 and https://github.com/ontoportal-lirmm/goo/issues/8