the generated file name is "b4bb8cee-d627-4f65-91c8-22e66e927683?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20220316%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220316T155722Z&X-Amz-Expires=300&X-Amz-Signature=fa9689e21c64f1d6b89ae12583d654763c3b821e2d6d6a0b9d8992416e1d025d&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=93400523&response-content-disposition=attachment%3B%20filename%3Defo.owl&response-content-type=application%2Foctet-stream"
untead of : "efo.owl"
Solution
Adding the regex for the case of github releases (and may be others)
Logs
[ontoportal@agroportal 20220316143509]$ bin/ncbo_ontology_process -o EFO
fatal: Not a git repository (or any of the parent directories): .git
(LD) >> Using rdf store localhost:8081
(LD) >> Using term search server at http://localhost:8983/solr/term_search_core1
(LD) >> Using property search server at http://localhost:8983/solr/prop_search_core1
(LD) >> Using HTTP Redis instance at localhost:6380
(LD) >> Using Goo Redis instance at localhost:6381
DEPRECATION: Rsolr.new/connect option `read_timeout` is deprecated and will be removed in Rsolr 3. `timeout` is currently a synonym, use that instead.
DEPRECATION: Rsolr.new/connect option `read_timeout` is deprecated and will be removed in Rsolr 3. `timeout` is currently a synonym, use that instead.
fatal: Not a git repository (or any of the parent directories): .git
(AN) >> Using ANN Redis instance at localhost:6379
(CR) >> Using Redis instance at localhost:6379
Processing the following tasks: {:process_rdf=>true, :index_search=>true, :index_properties=>true, :run_metrics=>true, :process_annotator=>true, :diff=>true, :params=>nil} on ontologies: ["EFO"]
Processing details are logged to STDOUT
I, [2022-03-16T16:58:31.258158 #12495] INFO -- : Logging parsing output to /srv/ontoportal/data/repository/EFO/42/parsing.log
I, [2022-03-16T16:58:31.258286 #12495] INFO -- : ["Starting to process http://data.bioontology.org/ontologies/EFO/submissions/42"]
D, [2022-03-16T16:58:31.258356 #12495] DEBUG -- : ["Pull location found, but no file in the upload file path. Retrying download."]
E, [2022-03-16T16:58:38.787479 #12495] ERROR -- : Failed, exception: Errno::ENAMETOOLONG: File name too long @ rb_sysopen - /srv/ontoportal/data/repository/EFO/42/b4bb8cee-d627-4f65-91c8-22e66e927683?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20220316%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220316T155722Z&X-Amz-Expires=300&X-Amz-Signature=fa9689e21c64f1d6b89ae12583d654763c3b821e2d6d6a0b9d8992416e1d025d&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=93400523&response-content-disposition=attachment%3B%20filename%3Defo.owl&response-content-type=application%2Foctet-stream
/usr/local/rbenv/versions/2.6.9/lib/ruby/2.6.0/fileutils.rb:1387:in `initialize'
/usr/local/rbenv/versions/2.6.9/lib/ruby/2.6.0/fileutils.rb:1387:in `open'
/usr/local/rbenv/versions/2.6.9/lib/ruby/2.6.0/fileutils.rb:1387:in `block in copy_file'
/usr/local/rbenv/versions/2.6.9/lib/ruby/2.6.0/fileutils.rb:1386:in `open'
/usr/local/rbenv/versions/2.6.9/lib/ruby/2.6.0/fileutils.rb:1386:in `copy_file'
/usr/local/rbenv/versions/2.6.9/lib/ruby/2.6.0/fileutils.rb:492:in `copy_file'
/usr/local/rbenv/versions/2.6.9/lib/ruby/2.6.0/fileutils.rb:419:in `block in cp'
/usr/local/rbenv/versions/2.6.9/lib/ruby/2.6.0/fileutils.rb:1558:in `block in fu_each_src_dest'
/usr/local/rbenv/versions/2.6.9/lib/ruby/2.6.0/fileutils.rb:1574:in `fu_each_src_dest0'
/usr/local/rbenv/versions/2.6.9/lib/ruby/2.6.0/fileutils.rb:1556:in `fu_each_src_dest'
/usr/local/rbenv/versions/2.6.9/lib/ruby/2.6.0/fileutils.rb:418:in `cp'
/srv/ontoportal/ncbo_cron_deployments/shared/bundle/ruby/2.6.0/bundler/gems/ontologies_linked_data-fc4d30bf7f57/lib/ontologies_linked_data/models/ontology_submission.rb:573:in `copy_file_repository'
/srv/ontoportal/ncbo_cron_deployments/releases/20220316143509/lib/ncbo_cron/ontology_submission_parser.rb:186:in `process_submission'
bin/ncbo_ontology_process:98:in `block in <main>'
bin/ncbo_ontology_process:81:in `each'
bin/ncbo_ontology_process:81:in `<main>'
syphax-bouazzouni
commented
2 years ago
Issue
There is an error in getting the filename int request response header. Exactly the regex isn't correct for github releases. (see https://github.com/ontoportal-lirmm/ontologies_linked_data/blob/master/lib/ontologies_linked_data/utils/file.rb#L142)
Use case
the pull location of EFO : https://github.com/EBISPOT/efo/releases/download/current/efo.owl
Solution
Adding the regex for the case of github releases (and may be others)
Logs