search

portagenetwork / roadmap

Developed by the the Alliance in collaboration with University of Alberta, DMP Assistant a data management planning tool, forking the DMP Roadmap codebase
MIT License
6 stars 1 forks source link

Net::OpenTimeout: execution expired #744

Open aaronskiba opened 1 month ago

aaronskiba commented 1 month ago

View details in Rollbar: https://app.rollbar.com/a/ualbertalib/fix/item/dmp_assistant/431


Net::OpenTimeout: execution expired
  File "/usr/lib64/ruby/2.7.0/net/http.rb", line 960, in initialize
  File "/usr/lib64/ruby/2.7.0/net/http.rb", line 960, in open
  File "/usr/lib64/ruby/2.7.0/net/http.rb", line 960, in block in connect
  File "/usr/lib64/ruby/2.7.0/timeout.rb", line 105, in timeout
  File "/usr/lib64/ruby/2.7.0/net/http.rb", line 958, in connect
  File "/usr/lib64/ruby/2.7.0/net/http.rb", line 943, in do_start
  File "/usr/lib64/ruby/2.7.0/net/http.rb", line 932, in start
  File "/usr/lib64/ruby/2.7.0/net/http.rb", line 1483, in request
  File "/var/www/sites/dmp/vendor/ruby/2.7.0/gems/httparty-0.21.0/lib/httparty/request.rb", line 156, in perform
  File "/var/www/sites/dmp/vendor/ruby/2.7.0/gems/httparty-0.21.0/lib/httparty.rb", line 612, in perform_request
  File "/var/www/sites/dmp/vendor/ruby/2.7.0/gems/httparty-0.21.0/lib/httparty.rb", line 526, in get
  File "/var/www/sites/dmp/vendor/ruby/2.7.0/gems/httparty-0.21.0/lib/httparty.rb", line 645, in get
  File "/var/www/sites/dmp/app/services/external_apis/base_service.rb", line 98, in http_get
  File "/var/www/sites/dmp/app/services/external_apis/re3data_service.rb", line 86, in query_re3data_repository
  File "/var/www/sites/dmp/app/services/external_apis/re3data_service.rb", line 58, in block in fetch
  File "/var/www/sites/dmp/vendor/ruby/2.7.0/gems/nokogiri-1.15.5-x86_64-linux/lib/nokogiri/xml/node_set.rb", line 235, in block in each
  File "/var/www/sites/dmp/vendor/ruby/2.7.0/gems/nokogiri-1.15.5-x86_64-linux/lib/nokogiri/xml/node_set.rb", line 234, in upto
  File "/var/www/sites/dmp/vendor/ruby/2.7.0/gems/nokogiri-1.15.5-x86_64-linux/lib/nokogiri/xml/node_set.rb", line 234, in each
  File "/var/www/sites/dmp/app/services/external_apis/re3data_service.rb", line 55, in fetch
  File "/var/www/sites/dmp/lib/tasks/utils/external_api.rake", line 14, in block (2 levels) in <top (required)>
  File "/var/www/sites/dmp/vendor/ruby/2.7.0/gems/rake-13.1.0/lib/rake/task.rb", line 281, in block in execute
  File "/var/www/sites/dmp/vendor/ruby/2.7.0/gems/rake-13.1.0/lib/rake/task.rb", line 281, in each
  File "/var/www/sites/dmp/vendor/ruby/2.7.0/gems/rake-13.1.0/lib/rake/task.rb", line 281, in execute
  File "/var/www/sites/dmp/vendor/ruby/2.7.0/gems/rake-13.1.0/lib/rake/task.rb", line 219, in block in invoke_with_call_chain
  File "/var/www/sites/dmp/vendor/ruby/2.7.0/gems/rake-13.1.0/lib/rake/task.rb", line 199, in synchronize
  File "/var/www/sites/dmp/vendor/ruby/2.7.0/gems/rake-13.1.0/lib/rake/task.rb", line 199, in invoke_with_call_chain
  File "/var/www/sites/dmp/vendor/ruby/2.7.0/gems/rake-13.1.0/lib/rake/task.rb", line 188, in invoke
  File "/var/www/sites/dmp/vendor/ruby/2.7.0/gems/rake-13.1.0/lib/rake/application.rb", line 182, in invoke_task
  File "/var/www/sites/dmp/vendor/ruby/2.7.0/gems/rake-13.1.0/lib/rake/application.rb", line 138, in block (2 levels) in top_level
  File "/var/www/sites/dmp/vendor/ruby/2.7.0/gems/rake-13.1.0/lib/rake/application.rb", line 138, in each
  File "/var/www/sites/dmp/vendor/ruby/2.7.0/gems/rake-13.1.0/lib/rake/application.rb", line 138, in block in top_level
  File "/var/www/sites/dmp/vendor/ruby/2.7.0/gems/rake-13.1.0/lib/rake/application.rb", line 147, in run_with_threads
  File "/var/www/sites/dmp/vendor/ruby/2.7.0/gems/rake-13.1.0/lib/rake/application.rb", line 132, in top_level
  File "/var/www/sites/dmp/vendor/ruby/2.7.0/gems/rake-13.1.0/lib/rake/application.rb", line 83, in block in run
  File "/var/www/sites/dmp/vendor/ruby/2.7.0/gems/rake-13.1.0/lib/rake/application.rb", line 208, in standard_exception_handling
  File "/var/www/sites/dmp/vendor/ruby/2.7.0/gems/rake-13.1.0/lib/rake/application.rb", line 80, in run
  File "/var/www/sites/dmp/vendor/ruby/2.7.0/gems/rake-13.1.0/exe/rake", line 27, in <top (required)>
  File "/usr/bin/rake", line 23, in load
  File "/usr/bin/rake", line 23, in <top (required)>
  File "/usr/lib64/ruby/gems/2.7.0/gems/bundler-2.4.15/lib/bundler/cli/exec.rb", line 58, in load
  File "/usr/lib64/ruby/gems/2.7.0/gems/bundler-2.4.15/lib/bundler/cli/exec.rb", line 58, in kernel_load
  File "/usr/lib64/ruby/gems/2.7.0/gems/bundler-2.4.15/lib/bundler/cli/exec.rb", line 23, in run
  File "/usr/lib64/ruby/gems/2.7.0/gems/bundler-2.4.15/lib/bundler/cli.rb", line 492, in exec
  File "/usr/lib64/ruby/gems/2.7.0/gems/bundler-2.4.15/lib/bundler/vendor/thor/lib/thor/command.rb", line 27, in run
  File "/usr/lib64/ruby/gems/2.7.0/gems/bundler-2.4.15/lib/bundler/vendor/thor/lib/thor/invocation.rb", line 127, in invoke_command
  File "/usr/lib64/ruby/gems/2.7.0/gems/bundler-2.4.15/lib/bundler/vendor/thor/lib/thor.rb", line 392, in dispatch
  File "/usr/lib64/ruby/gems/2.7.0/gems/bundler-2.4.15/lib/bundler/cli.rb", line 34, in dispatch
  File "/usr/lib64/ruby/gems/2.7.0/gems/bundler-2.4.15/lib/bundler/vendor/thor/lib/thor/base.rb", line 485, in start
  File "/usr/lib64/ruby/gems/2.7.0/gems/bundler-2.4.15/lib/bundler/cli.rb", line 28, in start
  File "/usr/lib64/ruby/gems/2.7.0/gems/bundler-2.4.15/exe/bundle", line 37, in block in <top (required)>
  File "/usr/lib64/ruby/gems/2.7.0/gems/bundler-2.4.15/lib/bundler/friendly_errors.rb", line 117, in with_friendly_errors
  File "/usr/lib64/ruby/gems/2.7.0/gems/bundler-2.4.15/exe/bundle", line 29, in <top (required)>
  File "/usr/bin/bundle", line 23, in load
  File "/usr/bin/bundle", line 23, in <main>
aaronskiba commented 1 month ago 
// lib/tasks/utils/external_api.rake (lines 10-15)
  desc 'Load Repositories from re3data'
  task load_re3data_repos: :environment do
    p 'Fetching the latest re3data repository metadata and updating the repositories table'
    p 'This can take in excess of 10 minutes to complete ...'
    ExternalApis::Re3dataService.fetch
  end

https://www.re3data.org/api/doc

aaronskiba commented 1 month ago

rails external_api:load_re3data_repos executed successfully on my local dev instance. However, it took over 35 min to complete.

aaronskiba commented 1 month ago

We should also consider removing the add_field_of_science_to_research_domains rake task within lib/tasks/utils/external_api.rake. The task was used to first populate the research_domains db table, but no longer seems to serves a purpose.