ncbo / ontologies_linked_data

Models and serializers for ontologies and related artifacts backed by 4store
Other
18 stars 8 forks source link

ontology processing errors #219

Open alexskr opened 1 week ago

alexskr commented 1 week ago

We are encountering errors when processing some ontologies. EFO:

E, [2024-10-09T10:24:23.594106 #365177] ERROR -- : ["Thread 1: NoMethodError: undefined method `map' for #<RDF::Literal::Decimal:0x819ee4(\"92.1\"^^<http://www.w3.org/2001/XMLSchema#decimal>)>\nDid you mean?  tap after retrying 10 times...\n/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/goo-74a012eebb94/lib/goo/base/resource.rb:250:in `block in map_attributes'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/goo-74a012eebb94/lib/goo/base/resource.rb:222:in `each'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/goo-74a012eebb94/lib/goo/base/resource.rb:222:in `map_attributes'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-8572cd25722c/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:132:in `rescue in block (4 levels) in index'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-8572cd25722c/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:118:in `block (4 levels) in index'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-8572cd25722c/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:117:in `each'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-8572cd25722c/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:117:in `block (3 levels) in index'"]
#<Thread:0x000055c56730a980 /srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-8572cd25722c/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:56 run> terminated with exception (report_on_exception is true):
/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/goo-74a012eebb94/lib/goo/base/settings/settings.rb:300:in `block in shape_attribute': Attribute `synonym` is not loaded for http://www.ebi.ac.uk/efo/EFO_0022503. Loaded attributes: #<Set: {:submission, :label, :prefLabel}>. (Goo::Base::AttributeNotLoaded)
    from /srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-8572cd25722c/lib/ontologies_linked_data/utils/ontology_csv_writer.rb:44:in `write_class'
    from /srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-8572cd25722c/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:148:in `block (5 levels) in index'
    from /srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-8572cd25722c/lib/ontologies_linked_data/models/ontology_submission.rb:90:in `synchronize'
    from /srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-8572cd25722c/lib/ontologies_linked_data/models/ontology_submission.rb:90:in `synchronize'
    from /srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-8572cd25722c/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:147:in `block (4 levels) in index'
    from /srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-8572cd25722c/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:117:in `each'
    from /srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-8572cd25722c/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:117:in `block (3 levels) in index'
E, [2024-10-09T10:24:23.596405 #365177] ERROR -- : ["\n\nGoo::Base::AttributeNotLoaded: Attribute `synonym` is not loaded for http://www.ebi.ac.uk/efo/EFO_0022503. Loaded attributes: #<Set: {:submission, :label, :prefLabel}>.\n"]
E, [2024-10-09T10:24:23.596490 #365177] ERROR -- : [["/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/goo-74a012eebb94/lib/goo/base/settings/settings.rb:300:in `block in shape_attribute'", "/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-8572cd25722c/lib/ontologies_linked_data/utils/ontology_csv_writer.rb:44:in `write_class'", "/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-8572cd25722c/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:148:in `block (5 levels) in index'", "/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-8572cd25722c/lib/ontologies_linked_data/models/ontology_submission.rb:90:in `synchronize'", "/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-8572cd25722c/lib/ontologies_linked_data/models/ontology_submission.rb:90:in `synchronize'", "/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-8572cd25722c/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:147:in `block (4 levels) in index'", "/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-8572cd25722c/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:117:in `each'", "/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-8572cd25722c/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:117:in `block (3 levels) in index'"]]
E, [2024-10-09T10:24:23.597308 #365177] ERROR -- : ["Goo::Base::AttributeNotLoaded: Attribute `synonym` is not loaded for http://www.ebi.ac.uk/efo/EFO_0022503. Loaded attributes: #<Set: {:submission, :label, :prefLabel}>.\n/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/goo-74a012eebb94/lib/goo/base/settings/settings.rb:300:in `block in shape_attribute'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-8572cd25722c/lib/ontologies_linked_data/utils/ontology_csv_writer.rb:44:in `write_class'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-8572cd25722c/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:148:in `block (5 levels) in index'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-8572cd25722c/lib/ontologies_linked_data/models/ontology_submission.rb:90:in `synchronize'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-8572cd25722c/lib/ontologies_linked_data/models/ontology_submission.rb:90:in `synchronize'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-8572cd25722c/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:147:in `block (4 levels) in index'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-8572cd25722c/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:117:in `each'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-8572cd25722c/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:117:in `block (3 levels) in index'"]
E, [2024-10-09T12:05:38.508345 #369858] ERROR -- : [["Failed caching classes for EFO (http://data.bioontology.org/ontologies/EFO/submissions/268)"]]
E, [2024-10-09T12:05:38.509743 #369858] ERROR -- : [["undefined method `casecmp' for 0.921e2:BigDecimal\n/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ncbo_annotator-b2731c98bfef/lib/ncbo_annotator.rb:343:in `block (3 levels) in create_term_cache_for_submission'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ncbo_annotator-b2731c98bfef/lib/ncbo_annotator.rb:336:in `each'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ncbo_annotator-b2731c98bfef/lib/ncbo_annotator.rb:336:in `block (2 levels) in create_term_cache_for_submission'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ncbo_annotator-b2731c98bfef/lib/ncbo_annotator.rb:292:in `each'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ncbo_annotator-b2731c98bfef/lib/ncbo_annotator.rb:292:in `block in create_term_cache_for_submission'\n\t/usr/local/rbenv/versions/3.0.6/lib/ruby/3.0.0/benchmark.rb:308:in `realtime'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ncbo_annotator-b2731c98bfef/lib/ncbo_annotator.rb:273:in `create_term_cache_for_submission'\n\t/srv/ontoportal/ncbo_cron/lib/ncbo_cron/ontology_submission_parser.rb:219:in `process_annotator'\n\t/srv/ontoportal/ncbo_cron/lib/ncbo_cron/ontology_submission_parser.rb:176:in `process_submission'\n\tbin/ncbo_ontology_process:106:in `block in <top (required)>'\n\tbin/ncbo_ontology_process:89:in `each'\n\tbin/ncbo_ontology_process:89:in `<top (required)>'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/gems/bundler-2.5.20/lib/bundler/cli/exec.rb:58:in `load'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/gems/bundler-2.5.20/lib/bundler/cli/exec.rb:58:in `kernel_load'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/gems/bundler-2.5.20/lib/bundler/cli/exec.rb:23:in `run'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/gems/bundler-2.5.20/lib/bundler/cli.rb:455:in `exec'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/gems/bundler-2.5.20/lib/bundler/vendor/thor/lib/thor/command.rb:28:in `run'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/gems/bundler-2.5.20/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/gems/bundler-2.5.20/lib/bundler/vendor/thor/lib/thor.rb:527:in `dispatch'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/gems/bundler-2.5.20/lib/bundler/cli.rb:35:in `dispatch'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/gems/bundler-2.5.20/lib/bundler/vendor/thor/lib/thor/base.rb:584:in `start'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/gems/bundler-2.5.20/lib/bundler/cli.rb:29:in `start'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/gems/bundler-2.5.20/exe/bundle:28:in `block in <top (required)>'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/gems/bundler-2.5.20/lib/bundler/friendly_errors.rb:117:in `with_friendly_errors'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/gems/bundler-2.5.20/exe/bundle:20:in `<top (required)>'\n\t/usr/local/rbenv/versions/3.0.6/bin/bundle:25:in `load'\n\t/usr/local/rbenv/versions/3.0.6/bin/bundle:25:in `<main>'"]]

FYPO:

I, [2024-10-09T13:30:40.046055 #380428]  INFO -- : ["Thread 1: Page 4 of 17 - 1000 ontology terms indexed in 45.203980855 sec."]
I, [2024-10-09T13:30:44.330454 #380428]  INFO -- : ["Thread 1: Page 5 of 17 - 1000 ontology terms retrieved in 9.737e-06 sec."]
E, [2024-10-09T13:30:44.772420 #380428] ERROR -- : ["Thread 1: Exception while mapping attributes for http://purl.obolibrary.org/obo/FYPO_0008033. Retrying 1 times..."]
E, [2024-10-09T13:30:46.773008 #380428] ERROR -- : ["Thread 1: Exception while mapping attributes for http://purl.obolibrary.org/obo/FYPO_0008033. Retrying 2 times..."]
E, [2024-10-09T13:30:48.773649 #380428] ERROR -- : ["Thread 1: Exception while mapping attributes for http://purl.obolibrary.org/obo/FYPO_0008033. Retrying 3 times..."]
E, [2024-10-09T13:30:50.774253 #380428] ERROR -- : ["Thread 1: Exception while mapping attributes for http://purl.obolibrary.org/obo/FYPO_0008033. Retrying 4 times..."]
E, [2024-10-09T13:30:52.774852 #380428] ERROR -- : ["Thread 1: Exception while mapping attributes for http://purl.obolibrary.org/obo/FYPO_0008033. Retrying 5 times..."]
E, [2024-10-09T13:30:54.775442 #380428] ERROR -- : ["Thread 1: Exception while mapping attributes for http://purl.obolibrary.org/obo/FYPO_0008033. Retrying 6 times..."]
E, [2024-10-09T13:30:56.776063 #380428] ERROR -- : ["Thread 1: Exception while mapping attributes for http://purl.obolibrary.org/obo/FYPO_0008033. Retrying 7 times..."]
E, [2024-10-09T13:30:58.776652 #380428] ERROR -- : ["Thread 1: Exception while mapping attributes for http://purl.obolibrary.org/obo/FYPO_0008033. Retrying 8 times..."]
E, [2024-10-09T13:31:00.777252 #380428] ERROR -- : ["Thread 1: Exception while mapping attributes for http://purl.obolibrary.org/obo/FYPO_0008033. Retrying 9 times..."]
E, [2024-10-09T13:31:02.777867 #380428] ERROR -- : ["Thread 1: Exception while mapping attributes for http://purl.obolibrary.org/obo/FYPO_0008033. Retrying 10 times..."]
E, [2024-10-09T13:31:04.778491 #380428] ERROR -- : ["Thread 1: Error mapping attributes for http://purl.obolibrary.org/obo/FYPO_0008033:"]
E, [2024-10-09T13:31:04.779553 #380428] ERROR -- : ["Thread 1: NoMethodError: undefined method `map' for #<RDF::Literal:0x24813c(\"A cell morphology phenotype in which a vegetative cell is elongated, has no septum, and contains one nucleus, and is arrested in mototic S-phase\"^^<http://www.w3.org/2001/XMLSchema#anyURI>)>\nDid you mean?  tap after retrying 10 times...\n/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/goo-f943ac33e87c/lib/goo/base/resource.rb:250:in `block in map_attributes'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/goo-f943ac33e87c/lib/goo/base/resource.rb:222:in `each'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/goo-f943ac33e87c/lib/goo/base/resource.rb:222:in `map_attributes'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-9ab4be437ed8/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:132:in `rescue in block (4 levels) in index'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-9ab4be437ed8/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:118:in `block (4 levels) in index'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-9ab4be437ed8/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:117:in `each'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-9ab4be437ed8/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:117:in `block (3 levels) in index'"]
#<Thread:0x000055997f310470 /srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-9ab4be437ed8/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:56 run> terminated with exception (report_on_exception is true):
/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/goo-f943ac33e87c/lib/goo/base/settings/settings.rb:300:in `block in shape_attribute': Attribute `definition` is not loaded for http://purl.obolibrary.org/obo/FYPO_0008033. Loaded attributes: #<Set: {:submission, :label, :prefLabel, :synonym}>. (Goo::Base::AttributeNotLoaded)
    from /srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-9ab4be437ed8/lib/ontologies_linked_data/utils/ontology_csv_writer.rb:48:in `write_class'
    from /srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-9ab4be437ed8/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:148:in `block (5 levels) in index'
    from /srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-9ab4be437ed8/lib/ontologies_linked_data/models/ontology_submission.rb:90:in `synchronize'
    from /srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-9ab4be437ed8/lib/ontologies_linked_data/models/ontology_submission.rb:90:in `synchronize'
    from /srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-9ab4be437ed8/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:147:in `block (4 levels) in index'
    from /srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-9ab4be437ed8/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:117:in `each'
    from /srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-9ab4be437ed8/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:117:in `block (3 levels) in index'
E, [2024-10-09T13:31:04.781787 #380428] ERROR -- : ["\n\nGoo::Base::AttributeNotLoaded: Attribute `definition` is not loaded for http://purl.obolibrary.org/obo/FYPO_0008033. Loaded attributes: #<Set: {:submission, :label, :prefLabel, :synonym}>.\n"]
E, [2024-10-09T13:31:04.781867 #380428] ERROR -- : [["/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/goo-f943ac33e87c/lib/goo/base/settings/settings.rb:300:in `block in shape_attribute'", "/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-9ab4be437ed8/lib/ontologies_linked_data/utils/ontology_csv_writer.rb:48:in `write_class'", "/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-9ab4be437ed8/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:148:in `block (5 levels) in index'", "/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-9ab4be437ed8/lib/ontologies_linked_data/models/ontology_submission.rb:90:in `synchronize'", "/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-9ab4be437ed8/lib/ontologies_linked_data/models/ontology_submission.rb:90:in `synchronize'", "/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-9ab4be437ed8/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:147:in `block (4 levels) in index'", "/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-9ab4be437ed8/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:117:in `each'", "/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-9ab4be437ed8/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:117:in `block (3 levels) in index'"]]
E, [2024-10-09T13:31:04.782032 #380428] ERROR -- : ["Goo::Base::AttributeNotLoaded: Attribute `definition` is not loaded for http://purl.obolibrary.org/obo/FYPO_0008033. Loaded attributes: #<Set: {:submission, :label, :prefLabel, :synonym}>.\n/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/goo-f943ac33e87c/lib/goo/base/settings/settings.rb:300:in `block in shape_attribute'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-9ab4be437ed8/lib/ontologies_linked_data/utils/ontology_csv_writer.rb:48:in `write_class'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-9ab4be437ed8/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:148:in `block (5 levels) in index'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-9ab4be437ed8/lib/ontologies_linked_data/models/ontology_submission.rb:90:in `synchronize'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-9ab4be437ed8/lib/ontologies_linked_data/models/ontology_submission.rb:90:in `synchronize'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-9ab4be437ed8/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:147:in `block (4 levels) in index'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-9ab4be437ed8/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:117:in `each'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-9ab4be437ed8/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:117:in `block (3 levels) in index'"]

STATO:

I, [2024-07-19T18:17:01.374473 #1163655]  INFO -- : ["Removed ontology terms index (0.126251334s)"]
I, [2024-07-19T18:17:39.346740 #1163655]  INFO -- : ["Thread 1: Page 1 of 1 - 959 ontology terms retrieved in 1.4692e-05 sec."]
E, [2024-07-19T18:17:39.849696 #1163655] ERROR -- : ["Thread 1: Exception while mapping attributes for http://purl.obolibrary.org/obo/STATO_0000594. Retrying 1 times..."]
E, [2024-07-19T18:17:41.850354 #1163655] ERROR -- : ["Thread 1: Exception while mapping attributes for http://purl.obolibrary.org/obo/STATO_0000594. Retrying 2 times..."]
E, [2024-07-19T18:17:43.851010 #1163655] ERROR -- : ["Thread 1: Exception while mapping attributes for http://purl.obolibrary.org/obo/STATO_0000594. Retrying 3 times..."]
E, [2024-07-19T18:17:45.851675 #1163655] ERROR -- : ["Thread 1: Exception while mapping attributes for http://purl.obolibrary.org/obo/STATO_0000594. Retrying 4 times..."]
E, [2024-07-19T18:17:47.852278 #1163655] ERROR -- : ["Thread 1: Exception while mapping attributes for http://purl.obolibrary.org/obo/STATO_0000594. Retrying 5 times..."]
E, [2024-07-19T18:17:49.852885 #1163655] ERROR -- : ["Thread 1: Exception while mapping attributes for http://purl.obolibrary.org/obo/STATO_0000594. Retrying 6 times..."]
E, [2024-07-19T18:17:51.853452 #1163655] ERROR -- : ["Thread 1: Exception while mapping attributes for http://purl.obolibrary.org/obo/STATO_0000594. Retrying 7 times..."]
E, [2024-07-19T18:17:53.854119 #1163655] ERROR -- : ["Thread 1: Exception while mapping attributes for http://purl.obolibrary.org/obo/STATO_0000594. Retrying 8 times..."]
E, [2024-07-19T18:17:55.854794 #1163655] ERROR -- : ["Thread 1: Exception while mapping attributes for http://purl.obolibrary.org/obo/STATO_0000594. Retrying 9 times..."]
E, [2024-07-19T18:17:57.855411 #1163655] ERROR -- : ["Thread 1: Exception while mapping attributes for http://purl.obolibrary.org/obo/STATO_0000594. Retrying 10 times..."]
E, [2024-07-19T18:17:59.856065 #1163655] ERROR -- : ["Thread 1: Error mapping attributes for http://purl.obolibrary.org/obo/STATO_0000594:"]
E, [2024-07-19T18:17:59.856953 #1163655] ERROR -- : ["Thread 1: NoMethodError: undefined method `map' for #<RDF::URI:0x46bcc(http://purl.obolibrary.org/obo/IAO_0000122)>\nDid you mean?  tap after retrying 10 times...\n/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/2.7.0/bundler/gems/goo-2cec53c2d841/lib/goo/mixins/sparql_client.rb:62:in `method_missing'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/2.7.0/bundler/gems/goo-2cec53c2d841/lib/goo/base/resource.rb:265:in `block in map_attributes'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/2.7.0/bundler/gems/goo-2cec53c2d841/lib/goo/base/resource.rb:237:in `each'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/2.7.0/bundler/gems/goo-2cec53c2d841/lib/goo/base/resource.rb:237:in `map_attributes'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/2.7.0/bundler/gems/ontologies_linked_data-70fc2c8b377b/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:132:in `rescue in block (4 levels) in index'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/2.7.0/bundler/gems/ontologies_linked_data-70fc2c8b377b/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:118:in `block (4 levels) in index'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/2.7.0/bundler/gems/ontologies_linked_data-70fc2c8b377b/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:117:in `each'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/2.7.0/bundler/gems/ontologies_linked_data-70fc2c8b377b/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:117:in `block (3 levels) in index'"]
E, [2024-07-19T18:17:59.858948 #1163655] ERROR -- : ["\n\nGoo::Base::AttributeNotLoaded: Attribute `synonym` is not loaded for http://purl.obolibrary.org/obo/STATO_0000594. Loaded attributes: #<Set: {:submission, :label, :prefLabel}>.\n"]
E, [2024-07-19T18:17:59.859411 #1163655] ERROR -- : [["/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/2.7.0/bundler/gems/goo-2cec53c2d841/lib/goo/base/settings/settings.rb:283:in `block in shape_attribute'", "/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/2.7.0/bundler/gems/ontologies_linked_data-70fc2c8b377b/lib/ontologies_linked_data/utils/ontology_csv_writer.rb:44:in `write_class'", "/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/2.7.0/bundler/gems/ontologies_linked_data-70fc2c8b377b/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:148:in `block (5 levels) in index'", "/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/2.7.0/bundler/gems/ontologies_linked_data-70fc2c8b377b/lib/ontologies_linked_data/models/ontology_submission.rb:90:in `synchronize'", "/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/2.7.0/bundler/gems/ontologies_linked_data-70fc2c8b377b/lib/ontologies_linked_data/models/ontology_submission.rb:90:in `synchronize'", "/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/2.7.0/bundler/gems/ontologies_linked_data-70fc2c8b377b/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:147:in `block (4 levels) in index'", "/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/2.7.0/bundler/gems/ontologies_linked_data-70fc2c8b377b/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:117:in `each'", "/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/2.7.0/bundler/gems/ontologies_linked_data-70fc2c8b377b/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:117:in `block (3 levels) in index'"]]
E, [2024-07-19T18:17:59.860244 #1163655] ERROR -- : ["Goo::Base::AttributeNotLoaded: Attribute `synonym` is not loaded for http://purl.obolibrary.org/obo/STATO_0000594. Loaded attributes: #<Set: {:submission, :label, :prefLabel}>.\n/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/2.7.0/bundler/gems/goo-2cec53c2d841/lib/goo/base/settings/settings.rb:283:in `block in shape_attribute'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/2.7.0/bundler/gems/ontologies_linked_data-70fc2c8b377b/lib/ontologies_linked_data/utils/ontology_csv_writer.rb:44:in `write_class'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/2.7.0/bundler/gems/ontologies_linked_data-70fc2c8b377b/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:148:in `block (5 levels) in index'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/2.7.0/bundler/gems/ontologies_linked_data-70fc2c8b377b/lib/ontologies_linked_data/models/ontology_submission.rb:90:in `synchronize'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/2.7.0/bundler/gems/ontologies_linked_data-70fc2c8b377b/lib/ontologies_linked_data/models/ontology_submission.rb:90:in `synchronize'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/2.7.0/bundler/gems/ontologies_linked_data-70fc2c8b377b/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:147:in `block (4 levels) in index'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/2.7.0/bundler/gems/ontologies_linked_data-70fc2c8b377b/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:117:in `each'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/2.7.0/bundler/gems/ontologies_linked_data-70fc2c8b377b/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:117:in `block (3 levels) in index'"]

ECTO:

E, [2024-10-08T13:22:30.388995 #332024] ERROR -- : ["Thread 1: Exception while mapping attributes for http://purl.obolibrary.org/obo/FOODON_00001264. Retrying 1 times..."]
E, [2024-10-08T13:22:32.389652 #332024] ERROR -- : ["Thread 1: Exception while mapping attributes for http://purl.obolibrary.org/obo/FOODON_00001264. Retrying 2 times..."]
E, [2024-10-08T13:22:34.390256 #332024] ERROR -- : ["Thread 1: Exception while mapping attributes for http://purl.obolibrary.org/obo/FOODON_00001264. Retrying 3 times..."]
E, [2024-10-08T13:22:36.390788 #332024] ERROR -- : ["Thread 1: Exception while mapping attributes for http://purl.obolibrary.org/obo/FOODON_00001264. Retrying 4 times..."]
E, [2024-10-08T13:22:38.391335 #332024] ERROR -- : ["Thread 1: Exception while mapping attributes for http://purl.obolibrary.org/obo/FOODON_00001264. Retrying 5 times..."]
E, [2024-10-08T13:22:40.391908 #332024] ERROR -- : ["Thread 1: Exception while mapping attributes for http://purl.obolibrary.org/obo/FOODON_00001264. Retrying 6 times..."]
E, [2024-10-08T13:22:42.392698 #332024] ERROR -- : ["Thread 1: Exception while mapping attributes for http://purl.obolibrary.org/obo/FOODON_00001264. Retrying 7 times..."]
E, [2024-10-08T13:22:44.393286 #332024] ERROR -- : ["Thread 1: Exception while mapping attributes for http://purl.obolibrary.org/obo/FOODON_00001264. Retrying 8 times..."]
E, [2024-10-08T13:22:46.393928 #332024] ERROR -- : ["Thread 1: Exception while mapping attributes for http://purl.obolibrary.org/obo/FOODON_00001264. Retrying 9 times..."]
E, [2024-10-08T13:22:48.394574 #332024] ERROR -- : ["Thread 1: Exception while mapping attributes for http://purl.obolibrary.org/obo/FOODON_00001264. Retrying 10 times..."]
E, [2024-10-08T13:22:50.395662 #332024] ERROR -- : ["Thread 1: Error mapping attributes for http://purl.obolibrary.org/obo/FOODON_00001264:"]
E, [2024-10-08T13:22:50.399137 #332024] ERROR -- : ["Thread 1: NoMethodError: undefined method `map' for #<RDF::Literal:0x63ec(\"http://purl.obolibrary.org/obo/FOODON_03411254\"^^<http://www.w3.org/2001/XMLSchema#anyURI>)>\nDid you mean?  tap after retrying 10 times...\n/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/goo-f943ac33e87c/lib/goo/base/resource.rb:250:in `block in map_attributes'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/goo-f943ac33e87c/lib/goo/base/resource.rb:222:in `each'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/goo-f943ac33e87c/lib/goo/base/resource.rb:222:in `map_attributes'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-9ab4be437ed8/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:132:in `rescue in block (4 levels) in index'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-9ab4be437ed8/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:118:in `block (4 levels) in index'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-9ab4be437ed8/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:117:in `each'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-9ab4be437ed8/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:117:in `block (3 levels) in index'"]
E, [2024-10-08T13:22:50.401899 #332024] ERROR -- : ["\n\nGoo::Base::AttributeNotLoaded: Attribute `synonym` is not loaded for http://purl.obolibrary.org/obo/FOODON_00001264. Loaded attributes: #<Set: {:submission, :label, :prefLabel}>.\n"]
E, [2024-10-08T13:22:50.401992 #332024] ERROR -- : [["/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/goo-f943ac33e87c/lib/goo/base/settings/settings.rb:300:in `block in shape_attribute'", "/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-9ab4be437ed8/lib/ontologies_linked_data/utils/ontology_csv_writer.rb:44:in `write_class'", "/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-9ab4be437ed8/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:148:in `block (5 levels) in index'", "/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-9ab4be437ed8/lib/ontologies_linked_data/models/ontology_submission.rb:90:in `synchronize'", "/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-9ab4be437ed8/lib/ontologies_linked_data/models/ontology_submission.rb:90:in `synchronize'", "/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-9ab4be437ed8/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:147:in `block (4 levels) in index'", "/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-9ab4be437ed8/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:117:in `each'", "/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-9ab4be437ed8/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:117:in `block (3 levels) in index'"]]
E, [2024-10-08T13:22:50.402106 #332024] ERROR -- : ["Goo::Base::AttributeNotLoaded: Attribute `synonym` is not loaded for http://purl.obolibrary.org/obo/FOODON_00001264. Loaded attributes: #<Set: {:submission, :label, :prefLabel}>.\n/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/goo-f943ac33e87c/lib/goo/base/settings/settings.rb:300:in `block in shape_attribute'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-9ab4be437ed8/lib/ontologies_linked_data/utils/ontology_csv_writer.rb:44:in `write_class'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-9ab4be437ed8/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:148:in `block (5 levels) in index'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-9ab4be437ed8/lib/ontologies_linked_data/models/ontology_submission.rb:90:in `synchronize'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-9ab4be437ed8/lib/ontologies_linked_data/models/ontology_submission.rb:90:in `synchronize'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-9ab4be437ed8/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:147:in `block (4 levels) in index'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-9ab4be437ed8/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:117:in `each'\n\t/srv/ontoportal/ncbo_cron/vendor/bundle/ruby/3.0.0/bundler/gems/ontologies_linked_data-9ab4be437ed8/lib/ontologies_linked_data/services/submission_process/operations/submission_indexer.rb:117:in `block (3 levels) in index'"]
I, [2024-10-08T13:22:50.502357 #332024]  INFO -- : ["Submission processing of http://data.bioontology.org/ontologies/ECTO/submissions/10 completed successfully"]
alexskr commented 1 week ago

relates to https://github.com/ncbo/bioportal_web_ui/issues/335 and https://github.com/ncbo/bioportal-project/issues/327

mdorf commented 1 week ago

@syphax-bouazzouni, The error happens in resource.rb (GOO) on line 250 because of a conflict introduced by the multilingual code:

              if include_languages && [RDF::URI, Hash].all?{|c| object.map(&:class).include?(c)}
                object = object.reduce({})  do |all, new_v|
                  new_v =  { none: [new_v] } if new_v.is_a?(RDF::URI)
                  all.merge(new_v) {|_, a, b| a + b }
                end
              elsif include_languages
=>              object = object.first
              end

The elsif condition turns the object from an Array to an Object. However, later in the code, there's no handling for when the object is NOT an Array. The method map is being called in the else block, which is a method of an Array:

            if object.is_a?(Hash)
              object = object.transform_values{|values| Array(values).map{|o|o.is_a?(RDF::URI) ? o : o.object}}
            else
=>            object = object.map {|o| o.is_a?(RDF::URI) ? o : o.object}
            end
alexskr commented 1 week ago

How do we reproduce this error so that we can catch it in our unit tests?

mdorf commented 6 days ago

@syphax-bouazzouni, some additional findings to help troubleshoot the issue:

When generating labels, for some reason the language information is NOT included in the unmapped property of each class:

@unmapped=
  {#<RDF::URI:0x238c(http://www.w3.org/2000/01/rdf-schema#label)>=>[#<RDF::Literal:0x23a0("Energy Metabolism"@en)>],
   #<RDF::URI:0x23b4(http://www.w3.org/2000/01/rdf-schema#comment)>=>
    [#<RDF::Literal:0x23c8("A set of metabolic pathways that are involved in the generation and use of energy within an organism, typically through processes like cellular
respiration and photosynthesis."@en)>],

However, when indexing, the language information IS included in unmapped of each class:

@unmapped=
  {#<RDF::URI:0x4fc4(http://www.w3.org/2000/01/rdf-schema#label)>=>{:en=>[#<RDF::Literal:0x4fd8("Energy Metabolism"@en)>]},
   #<RDF::URI:0x4fec(http://www.w3.org/2000/01/rdf-schema#comment)>=>
    {:en=>
      [#<RDF::Literal:0x5000("A set of metabolic pathways that are involved in the generation and use of energy within an organism, typically through processes like cellula
r respiration and photosynthesis."@en)>]},
mdorf commented 1 day ago

This has been fixed in #220