search

agroportal / project-management

Repository used to consolidate documentation about the AgroPortal project and track content related issues.
http://agroportal.lirmm.fr
7 stars 0 forks source link

CO_347 has no submission reachable #175

Closed jonquet closed 2 years ago

jonquet commented 2 years ago

http://data.agroportal.lirmm.fr/ontologies/CO_347/latest_submission returns an error.

Repository folder shows 25 submission. The ontology has been affected by the issue reported in #167 Certainly the last parsing had a problem and the ontology ended up with no submission http://data.agroportal.lirmm.fr/ontologies/CO_347/submissions/25 => fails http://data.agroportal.lirmm.fr/ontologies/CO_347/submissions/24 => work

I would suggest to remove 25 "manually" (using ontologies_api) and delete the files. It can't be done via REST API has the submission is not reachable.

syphax-bouazzouni commented 2 years ago

May be it's caused by the property : http://data.agroportal.lirmm.fr/submissions/?display=hasOntologyLanguage&display_links=false&display_context=false

syphax-bouazzouni commented 2 years ago

Backend resolution

Origin of the problem

From the /srv/ontoportal/ontologies_api/current/log/unicorn.stdout.log file we can see the following :

Bad data found in submission: 

{
   :"id=>""http://data.agroportal.lirmm.fr/ontologies/CO_347/submissions/25",
   :submissionId=>25,
   :"metrics=>nil",
   :"ontology=>"{
      :"administeredBy=>"[
         "http://data.agroportal.lirmm.fr/users/admin",
         "http://data.agroportal.lirmm.fr/users/Malaporte"
      ],
      :"acronym=>""CO_347",
      :"name=>""Castor Bean Ontology",
      :"summaryOnly=>nil",
      :"flat=>nil",
      :"ontologyType=>""http://data.agroportal.lirmm.fr/ontology_types/ONTOLOGY",
      :"group=>"[
         "http://data.agroportal.lirmm.fr/groups/CROP"
      ],
      :"hasDomain=>"[
         "http://data.agroportal.lirmm.fr/categories/TRAITS"
      ],
      :"viewingRestriction=>""public",
      :"viewOf=>nil",
      :"views=>"[

      ],
      "@id""=>""http://data.agroportal.lirmm.fr/ontologies/CO_347",
      "@type""=>""http://data.bioontology.org/metadata/Ontology",
      "links""=>"{
         "submissions""=>""http://data.agroportal.lirmm.fr/ontologies/CO_347/submissions",
         "properties""=>""http://data.agroportal.lirmm.fr/ontologies/CO_347/properties",
         "classes""=>""http://data.agroportal.lirmm.fr/ontologies/CO_347/classes",
         "single_class""=>""http://data.agroportal.lirmm.fr/ontologies/CO_347/classes/{class_id}",
         "roots""=>""http://data.agroportal.lirmm.fr/ontologies/CO_347/classes/roots",
         "instances""=>""http://data.agroportal.lirmm.fr/ontologies/CO_347/instances",
         "metrics""=>""http://data.agroportal.lirmm.fr/ontologies/CO_347/metrics",
         "reviews""=>""http://data.agroportal.lirmm.fr/ontologies/CO_347/reviews",
         "notes""=>""http://data.agroportal.lirmm.fr/ontologies/CO_347/notes",
         "groups""=>""http://data.agroportal.lirmm.fr/ontologies/CO_347/groups",
         "categories""=>""http://data.agroportal.lirmm.fr/ontologies/CO_347/categories",
         "latest_submission""=>""http://data.agroportal.lirmm.fr/ontologies/CO_347/latest_submission",
         "projects""=>""http://data.agroportal.lirmm.fr/ontologies/CO_347/projects",
         "download""=>""http://data.agroportal.lirmm.fr/ontologies/CO_347/download",
         "views""=>""http://data.agroportal.lirmm.fr/ontologies/CO_347/views",
         "analytics""=>""http://data.agroportal.lirmm.fr/ontologies/CO_347/analytics",
         "ui""=>""http://agroportal.lirmm.fr/ontologies/CO_347",
         "@context""=>"{
            "submissions""=>""http://data.bioontology.org/metadata/OntologySubmission",
            "properties""=>""http://data.bioontology.org/metadata/Property",
            "classes""=>""http://www.w3.org/2002/07/owl#Class",
            "single_class""=>""http://www.w3.org/2002/07/owl#Class",
            "roots""=>""http://www.w3.org/2002/07/owl#Class",
            "instances""=>""http://data.bioontology.org/metadata/Instance",
            "metrics""=>""http://data.bioontology.org/metadata/Metrics",
            "reviews""=>""http://data.bioontology.org/metadata/Review",
            "notes""=>""http://data.bioontology.org/metadata/Note",
            "groups""=>""http://data.bioontology.org/metadata/Group",
            "categories""=>""http://data.bioontology.org/metadata/Category",
            "latest_submission""=>""http://data.bioontology.org/metadata/OntologySubmission",
            "projects""=>""http://data.bioontology.org/metadata/Project",
            "download""=>""http://data.bioontology.org/metadata/Ontology",
            "views""=>""http://data.bioontology.org/metadata/Ontology",
            "analytics""=>""http://data.bioontology.org/metadata/Analytics",
            "ui""=>""http://data.bioontology.org/metadata/Ontology"
         }
      },
      "@context""=>"{
         "@vocab""=>""http://data.bioontology.org/metadata/",
         :"acronym=>""http://data.bioontology.org/metadata/omvacronym",
         :"name=>""http://data.bioontology.org/metadata/omvname",
         :"administeredBy=>"{
            "@id""=>""http://data.bioontology.org/metadata/User",
            "@type""=>""@id"
         },
         :"group=>"{
            "@id""=>""http://data.bioontology.org/metadata/Group",
            "@type""=>""@id"
         },
         :"hasDomain=>"{
            "@id""=>""http://data.bioontology.org/metadata/Category",
            "@type""=>""@id"
         },
         :"viewOf=>"{
            "@id""=>""http://data.bioontology.org/metadata/Ontology",
            "@type""=>""@id"
         },
         :"ontologyType=>"{
            "@id""=>""http://data.bioontology.org/metadata/OntologyType",
            "@type""=>""@id"
         }
      }
   },
   :"submissionStatus=>"[

   ],
   :"classType=>nil",
   :"hasOntologyLanguage=>nil",
   :"uploadFilePath=>nil",
   :"pullLocation=>nil",
   :"released=>nil",
   :"creationDate=>nil",
   :"status=>nil",
   :"contact=>"[

   ],
   :"prefLabelProperty=>nil",
   :"definitionProperty=>nil",
   :"synonymProperty=>nil",
   :"authorProperty=>nil",
   :"hierarchyProperty=>nil",
   :"obsoleteProperty=>nil",
   :"obsoleteParent=>nil",
   :"homepage=>nil",
   :"publication=>nil",
   :"URI=>nil",
   :"naturalLanguage=>"[

   ],
   :"documentation=>nil",
   :"version=>nil",
   :"description=>nil",
   :numberOfClasses=>344,
   :numberOfIndividuals=>0,
   :numberOfProperties=>3,
   :maxDepth=>3,
   :maxChildCount=>75,
   :averageChildCount=>10,
   :classesWithOneChild=>1,
   :classesWithMoreThan25Children=>3,
   :classesWithNoDefinition=>201,
   :"modificationDate=>nil",
   :"entities=>nil",
   :"numberOfAxioms=>nil",
   :"keyClasses=>nil",
   :"keywords=>""Castor Bean",
   :"knownUsage=>""The concepts of the CO are used to curate agronomic databases and describe the data.",
   :"notes=>nil",
   :"conformsToKnowledgeRepresentationParadigm=>nil",
   :"hasContributor=>nil",
   :"hasCreator=>nil",
   :"designedForOntologyTask=>"[

   ],
   :"wasGeneratedBy=>nil",
   :"wasInvalidatedBy=>nil",
   :"curatedBy=>nil",
   :"endorsedBy=>"[

   ],
   :"fundedBy=>""CGIAR (www.cgiar.org/)",
   :"translator=>nil",
   :"hasDomain=>""http://data.bioontology.org/categories/TRAITS",
   :"hasFormalityLevel=>""http://w3id.org/nkos/nkostype#ontology",
   :"hasLicense=>""https://creativecommons.org/licenses/by/4.0/",
   :"hasOntologySyntax=>""http://www.w3.org/ns/formats/RDF_XML",
   :"isOfType=>""http://omv.ontoware.org/2005/05/ontology#ApplicationOntology",
   :"usedOntologyEngineeringMethodology=>""http://www.cropontology.org/TD_template_v5.xls",
   :"usedOntologyEngineeringTool=>nil",
   :"useImports=>"[

   ],
   :"hasPriorVersion=>#<RDF"::"URI":"0x3702350(http"://data.agroportal.lirmm.fr/ontologies/CO_347/submissions/23)>,
   :"isBackwardCompatibleWith=>"[

   ],
   :"isIncompatibleWith=>"[

   ],
   :"deprecated=>false",
   :"versionIRI=>nil",
   :"ontologyRelatedTo=>"[
      "#<RDF"::"URI":"0x2fdb720(http":,
      "#<RDF"::"URI":"0x2fda85c(http":,
      "#<RDF"::"URI":"0x2fda1b8(http":
   ],
   :"comesFromTheSameDomain=>"[

   ],
   :"similarTo=>"[

   ],
   :"isAlignedTo=>"[
      "#<RDF"::"URI":"0x357ad0c(http":
   ],
   :"explanationEvolution=>nil",
   :"generalizes=>nil",
   :"hasDisparateModelling=>nil",
   :"hiddenLabel=>nil",
   :"coverage=>nil",
   :"publisher=>""CROP ONTOLOGY (http://www.cropontology.org/)",
   :"identifier=>""",
   :"source=>nil",
   :"abstract=>nil",
   :"alternative=>nil",
   :"hasPart=>nil",
   :"isFormatOf=>nil",
   :"hasFormat=>nil",
   :"audience=>nil",
   :"valid=>nil",
   :"accrualMethod=>nil",
   :"accrualPeriodicity=>nil",
   :"accrualPolicy=>nil",
   :"endpoint=>#<RDF"::"URI":"0x3fee874(http":,
   :"dataDump=>#<RDF"::"URI":"0x3fee75c(http"://data.agroportal.lirmm.fr/ontologies/CO_347/download?download_format=rdf)>,
   :"csvDump=>#<RDF"::"URI":"0x3fee4dc(http"://data.agroportal.lirmm.fr/ontologies/CO_347/download?download_format=csv)>,
   :"openSearchDescription=>#<RDF"::"URI":"0x3fee1a8(http"://data.agroportal.lirmm.fr/search?ontologies=CO_347&q=)>,
   :"uriLookupEndpoint=>#<RDF"::"URI":"0x403105c(http"://data.agroportal.lirmm.fr/search?ontologies=CO_347&require_exact_match=true&q=)>,
   :"uriRegexPattern=>nil",
   :"depiction=>nil",
   :"logo=>#<RDF"::"URI":"0x4033dd4(http":,
   :"competencyQuestion=>nil",
   :"usedBy=>"[

   ],
   :"metadataVoc=>"[

   ],
   :"hasDisjunctionsWith=>nil",
   :"toDoList=>nil",
   :"example=>nil",
   :"preferredNamespaceUri=>nil",
   :"preferredNamespacePrefix=>nil",
   :"morePermissions=>""Crop Ontology by Integrated Breeding Platform is licensed under a Creative Commons Attribution 4.0 International License.",
   :"useGuidelines=>nil",
   :"curatedOn=>nil",
   :"repository=>nil",
   :"bugDatabase=>nil",
   :"mailingList=>nil",
   :"exampleIdentifier=>#<RDF"::"URI":"0x40f1320(http"::0000108)>,
   :"award=>nil",
   :"copyrightHolder=>nil",
   :"associatedMedia=>nil",
   :"workTranslation=>nil",
   :"translationOfWork=>nil",
   :"includedInDataCatalog=>"[
      "#<RDF"::"URI":"0x412ca88(http"://www.cropontology.org/ontology/CO_347)>,
      "#<RDF"::"URI":"0x412c628(https"://www.ebi.ac.uk/ols/ontologies/co_347)>
   ],
   :"diffFilePath=>""/srv/ontoportal/data/repository/CO_347/24/bubastis_diff.xml",
   :"masterFileName=>nil",
   :"missingImports=>"[

   ]
}

Conclusion : This last submission is not valid , and have this errors :

:"hasOntologyLanguage=>"{
   :"existence=>""`hasOntologyLanguage` value cannot be nil"
},
:"released=>"{
   :"existence=>""`released` value cannot be nil"
},
:uploadFilePath=> ["In non-summary only submissions a data file or url must be provided."]

Fix

To solve this we had to update the submission in the console manually, like this : 

[root@agroportal ncbo_cron]# bin/ncbo_cron --console
[1] pry(main)> submission_id = "http://data.agroportal.lirmm.fr/ontologies/CO_347/submissions/25"
[2] pry(main)> sub = LinkedData::Models::OntologySubmission.find(RDF::IRI.new(submission_id)).first
[11] pry(main)> sub.bring_remaining
[13] pry(main)> sub.valid?
=> false
[14] pry(main)> sub.errors
=> show the errors
[25] pry(main)> sub.hasOntologyLanguage=LinkedData::Models::OntologyFormat.find("OWL").first
[26] pry(main)> sub.released=DateTime.now
[30] pry(main)> sub.uploadFilePath= "/srv/ontoportal/data/repository/CO_347/25/owl"
[31] pry(main)> sub.valid?
=> true
[32] pry(main)> sub.save

Result

the submission http://data.agroportal.lirmm.fr/ontologies/CO_347/submissions/25?display=all , is now reachable but still the UI is not working image

syphax-bouazzouni commented 2 years ago

UI resolution

Origin of the probelem in UI

in the log file we can see image Which says that we don't have creationDate

Solution

[root@agroportal ncbo_cron]# bin/ncbo_cron --console
[1] pry(main)> submission_id = "http://data.agroportal.lirmm.fr/ontologies/CO_347/submissions/25"
[2] pry(main)> sub = LinkedData::Models::OntologySubmission.find(RDF::IRI.new(submission_id)).first
[11] pry(main)> sub.bring_remaining
[49] pry(main)> sub.creationDate = sub.released
[50] pry(main)> sub.save

Result

UI is working image

syphax-bouazzouni commented 2 years ago

Content cleanup

The problem

The last submission of CO_347 is not parsed

Solution

[root@agroportal ncbo_cron]# bin/ncbo_ontology_process -o CO_347

Result

image