search

ncbo / ncbo_cron

Jobs that run on a regular basis in the NCBO infrastructure
Other
2 stars 6 forks source link

Failing to pull new versions of MONDO since April 2019 #23

Closed jvendetti closed 2 years ago

jvendetti commented 5 years ago

While investigating a processing issue with the MONDO ontology, I noticed that a new empty directory was created by the nightly ontology pull job for the latest version of MONDO that was never populated with the ontology source file or any parsing logs (/srv/ncbo/repository/MONDO/40). In other words, the system is trying to create a new submission (with ID 40), but the directory remains empty.

I checked the /srv/ncbo/ncbo_cron/logs/scheduler-pull.log file and found repeated occurrences of the following stack trace going back to April of this year:

I, [2019-04-10T18:14:35.420429 #18483]  INFO -- : New file found for MONDO
old: 5d801f6fd7f01ba5c387ad9b7dc52804
new: 32dcb0688ba8351f98c31e59d87d1829
E, [2019-04-10T18:14:35.489539 #18483] ERROR -- : Problem retrieving MONDO in OntologyPull:
File name too long @ rb_sysopen - /srv/ncbo/repository/MONDO/40/8c468600-5ac5-11e9-9c9e-ad6545090715?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20190411%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20190411T011428Z&X-Amz-Expires=300&X-Amz-Signature=30fd7e3d2f4aa3b709facee8903f8b4da79cce310eb5e407dcf6be39c2d23235&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Dmondo.owl&response-content-type=application%2Foctet-stream
/usr/local/rbenv/versions/2.5.3/lib/ruby/2.5.0/fileutils.rb:1292:in `initialize'
        /usr/local/rbenv/versions/2.5.3/lib/ruby/2.5.0/fileutils.rb:1292:in `open'
        /usr/local/rbenv/versions/2.5.3/lib/ruby/2.5.0/fileutils.rb:1292:in `block in copy_file'
        /usr/local/rbenv/versions/2.5.3/lib/ruby/2.5.0/fileutils.rb:1291:in `open'
        /usr/local/rbenv/versions/2.5.3/lib/ruby/2.5.0/fileutils.rb:1291:in `copy_file'
        /usr/local/rbenv/versions/2.5.3/lib/ruby/2.5.0/fileutils.rb:432:in `copy_file'
        /usr/local/rbenv/versions/2.5.3/lib/ruby/2.5.0/fileutils.rb:359:in `block in cp'
        /usr/local/rbenv/versions/2.5.3/lib/ruby/2.5.0/fileutils.rb:1463:in `block in fu_each_src_dest'
        /usr/local/rbenv/versions/2.5.3/lib/ruby/2.5.0/fileutils.rb:1479:in `fu_each_src_dest0'
        /usr/local/rbenv/versions/2.5.3/lib/ruby/2.5.0/fileutils.rb:1461:in `fu_each_src_dest'
        /usr/local/rbenv/versions/2.5.3/lib/ruby/2.5.0/fileutils.rb:358:in `cp'
        /srv/ncbo/ncbo_cron/vendor/bundle/ruby/2.5.0/bundler/gems/ontologies_linked_data-548e7b1e4fb8/lib/ontologies_linked_data/models/ontology_submission.rb:135:in `copy_file_repository'
        /srv/ncbo/ncbo_cron/lib/ncbo_cron/ontology_pull.rb:101:in `create_submission'
        /srv/ncbo/ncbo_cron/lib/ncbo_cron/ontology_pull.rb:67:in `block in do_remote_ontology_pull'
        /srv/ncbo/ncbo_cron/lib/ncbo_cron/ontology_pull.rb:29:in `each'
        /srv/ncbo/ncbo_cron/lib/ncbo_cron/ontology_pull.rb:29:in `do_remote_ontology_pull'
        /srv/ncbo/ncbo_cron/bin/ncbo_cron:261:in `block (3 levels) in <main>'
        /srv/ncbo/ncbo_cron/lib/ncbo_cron/scheduler.rb:65:in `block (3 levels) in scheduled_locking_job'
        /srv/ncbo/ncbo_cron/lib/ncbo_cron/scheduler.rb:51:in `fork'
        /srv/ncbo/ncbo_cron/lib/ncbo_cron/scheduler.rb:51:in `block (2 levels) in scheduled_locking_job'
        /srv/ncbo/ncbo_cron/vendor/bundle/ruby/2.5.0/gems/mlanett-redis-lock-0.2.7/lib/redis-lock.rb:43:in `lock'
        /srv/ncbo/ncbo_cron/vendor/bundle/ruby/2.5.0/gems/mlanett-redis-lock-0.2.7/lib/redis-lock.rb:234:in `lock'
        /srv/ncbo/ncbo_cron/lib/ncbo_cron/scheduler.rb:50:in `block in scheduled_locking_job'
        /srv/ncbo/ncbo_cron/vendor/bundle/ruby/2.5.0/gems/rufus-scheduler-2.0.24/lib/rufus/sc/jobs.rb:230:in `trigger_block'
        /srv/ncbo/ncbo_cron/vendor/bundle/ruby/2.5.0/gems/rufus-scheduler-2.0.24/lib/rufus/sc/jobs.rb:204:in `block in trigger'
        /srv/ncbo/ncbo_cron/vendor/bundle/ruby/2.5.0/gems/rufus-scheduler-2.0.24/lib/rufus/sc/scheduler.rb:430:in `block in trigger_job'
jvendetti commented 3 years ago

Probable duplicate of https://github.com/ncbo/ontologies_linked_data/issues/112

jvendetti commented 2 years ago

Fixed by https://github.com/ncbo/ontologies_linked_data/pull/134 from @syphax-bouazzouni. Released to production as part of https://github.com/ncbo/ontologies_api/releases/tag/v5.23.0.