AG: "Exception: could not `LANG=C grep -v '_:genid' /tmp/file_nobnodes..."

[alexskr]

A number of ontologies are not processing correctly in staging ENV with AG backend.

Ontologies: SCTO,SDPV,MDRFRE,etc.

E, [2023-06T14:14:49.851643 #26201] ERROR -- : ["Exception: could not `LANG=C grep -v '_:genid' /tmp/file_nobnodes20230306-26201-3hmvtr/data.nt > /tmp/file_nobnodes20230306-26201-3hmvtr/data_no_bnodes.nt`: \n/srv/ncbo/ncbo_cron/vendor/bundle/ruby/2.7.0/bundler/gems/goo-b8c7867450ec/lib/goo/sparql/client.rb:74:in `bnodes_filter_file'\n\t/srv/ncbo/ncbo_cron/vendor/bundle/ruby/2.7.0/bundler/gems/goo-b8c7867450ec/lib/goo/sparql/client.rb:90:in `append_triples_no_bnodes'\n\t/srv/ncbo/ncbo_cron/vendor/bundle/ruby/2.7.0/bundler/gems/goo-b8c7867450ec/lib/goo/sparql/client.rb:135:in `append_data_triples'\n\t/srv/ncbo/ncbo_cron/vendor/bundle/ruby/2.7.0/bundler/gems/goo-b8c7867450ec/lib/goo/sparql/client.rb:147:in `append_triples'\n\t/srv/ncbo/ncbo_cron/vendor/bundle/ruby/2.7.0/bundler/gems/ontologies_linked_data-0f16d91a9391/lib/ontologies_linked_data/models/ontology_submission.rb:750:in `generate_missing_labels_post'\n\t/srv/ncbo/ncbo_cron/vendor/bundle/ruby/2.7.0/bundler/gems/ontologies_linked_data-0f16d91a9391/lib/ontologies_linked_data/models/ontology_submission.rb:629:in `call'\n\t/srv/ncbo/ncbo_cron/vendor/bundle/ruby/2.7.0/bundler/gems/ontologies_linked_data-0f16d91a9391/lib/ontologies_linked_data/models/ontology_submission.rb:629:in `block (2 levels) in loop_classes'\n\t/srv/ncbo/ncbo_cron/vendor/bundle/ruby/2.7.0/bundler/gems/ontologies_linked_data-0f16d91a9391/lib/ontologies_linked_data/models/ontology_submission.rb:520:in `block in process_callbacks'\n\t/srv/ncbo/ncbo_cron/vendor/bundle/ruby/2.7.0/bundler/gems/ontologies_linked_data-0f16d91a9391/lib/ontologies_linked_data/models/ontology_submission.rb:516:in `delete_if'\n\t/srv/ncbo/ncbo_cron/vendor/bundle/ruby/2.7.0/bundler/gems/ontologies_linked_data-0f16d91a9391/lib/ontologies_linked_data/models/ontology_submission.rb:516:in `process_callbacks'\n\t/srv/ncbo/ncbo_cron/vendor/bundle/ruby/2.7.0/bundler/gems/ontologies_linked_data-0f16d91a9391/lib/ontologies_linked_data/models/ontology_submission.rb:628:in `block in loop_classes'\n\t/usr/local/rbenv/versions/2.7.6/lib/ruby/2.7.0/benchmark.rb:308:in `realtime'\n\t/srv/ncbo/ncbo_cron/vendor/bundle/ruby/2.7.0/bundler/gems/ontologies_linked_data-0f16d91a9391/lib/ontologies_linked_data/models/ontology_submission.rb:547:in `loop_classes'\n\t/srv/ncbo/ncbo_cron/vendor/bundle/ruby/2.7.0/bundler/gems/ontologies_linked_data-0f16d91a9391/lib/ontologies_linked_data/models/ontology_submission.rb:1052:in `process_submission'\n\t/srv/ncbo/ncbo_cron/lib/ncbo_cron/ontology_submission_parser.rb:178:in `process_submission'\n\tbin/ncbo_ontology_process:103:in `block in <top (required)>'\n\tbin/ncbo_ontology_process:86:in `each'\n\tbin/ncbo_ontology_process:86:in `<top (required)>'\n\t/usr/local/rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/bundler-2.3.12/lib/bundler/cli/exec.rb:58:in `load'\n\t/usr/local/rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/bundler-2.3.12/lib/bundler/cli/exec.rb:58:in `kernel_load'\n\t/usr/local/rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/bundler-2.3.12/lib/bundler/cli/exec.rb:23:in `run'\n\t/usr/local/rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/bundler-2.3.12/lib/bundler/cli.rb:483:in `exec'\n\t/usr/local/rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/bundler-2.3.12/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'\n\t/usr/local/rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/bundler-2.3.12/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'\n\t/usr/local/rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/bundler-2.3.12/lib/bundler/vendor/thor/lib/thor.rb:392:in `dispatch'\n\t/usr/local/rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/bundler-2.3.12/lib/bundler/cli.rb:31:in `dispatch'\n\t/usr/local/rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/bundler-2.3.12/lib/bundler/vendor/thor/lib/thor/base.rb:485:in `start'\n\t/usr/local/rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/bundler-2.3.12/lib/bundler/cli.rb:25:in `start'\n\t/usr/local/rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/bundler-2.3.12/exe/bundle:48:in `block in <top (required)>'\n\t/usr/local/rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/bundler-2.3.12/lib/bundler/friendly_errors.rb:103:in `with_friendly_errors'\n\t/usr/local/rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/bundler-2.3.12/exe/bundle:36:in `<top (required)>'\n\t/usr/local/rbenv/versions/2.7.6/bin/bundle:23:in `load'\n\t/usr/local/rbenv/versions/2.7.6/bin/bundle:23:in `<main>'"]

Temp directory /tmp/file_nobnodes20230306-26201-3hmvtr is not automatically removed and contains empty files data_no_bnodes.nt and data.nt

[alexskr]

I am able to process those ontologies in my clean dev environment if I import those ontologies individually but run into the same issue when I point my dev env to the staging AG server.

[alexskr]

this error can be found in the logs in the 4store/prod env so might not be related to AG

[alexskr]

Parsing still fails for the following ontologies with the latest fix: BILELTEST DSIP