search

Knowledge-Graph-Hub / kg-covid-19

An instance of KG Hub to produce a knowledge graph for COVID-19 response.
https://github.com/Knowledge-Graph-Hub/kg-covid-19/wiki
BSD 3-Clause "New" or "Revised" License
79 stars 26 forks source link

Add kghub downloader - test on Jenkins #449

Closed caufieldjh closed 2 years ago

sonarcloud[bot] commented 2 years ago

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

100.0% 100.0% Coverage
0.0% 0.0% Duplication

caufieldjh commented 2 years ago

The Elasticsearch parts of the download encounter this:

15:01:58  Traceback (most recent call last):
15:01:58    File "run.py", line 166, in <module>
15:01:58      cli()
15:01:58    File "/var/lib/jenkins/workspace/add_kghub_downloader_run_jenkins/gitrepo/venv/lib/python3.8/site-packages/click/core.py", line 829, in __call__
15:01:58      return self.main(*args, **kwargs)
15:01:58    File "/var/lib/jenkins/workspace/add_kghub_downloader_run_jenkins/gitrepo/venv/lib/python3.8/site-packages/click/core.py", line 782, in main
15:01:58      rv = self.invoke(ctx)
15:01:58    File "/var/lib/jenkins/workspace/add_kghub_downloader_run_jenkins/gitrepo/venv/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
15:01:58      return _process_result(sub_ctx.command.invoke(sub_ctx))
15:01:58    File "/var/lib/jenkins/workspace/add_kghub_downloader_run_jenkins/gitrepo/venv/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
15:01:58      return ctx.invoke(self.callback, **ctx.params)
15:01:58    File "/var/lib/jenkins/workspace/add_kghub_downloader_run_jenkins/gitrepo/venv/lib/python3.8/site-packages/click/core.py", line 610, in invoke
15:01:58      return callback(*args, **kwargs)
15:01:58    File "run.py", line 39, in download
15:01:58      kg_download(*args, **kwargs)
15:01:58    File "/var/lib/jenkins/workspace/add_kghub_downloader_run_jenkins/gitrepo/kg_covid_19/download.py", line 20, in download
15:01:58      download_from_yaml(yaml_file=yaml_file, output_dir=output_dir,
15:01:58    File "/var/lib/jenkins/workspace/add_kghub_downloader_run_jenkins/gitrepo/venv/lib/python3.8/site-packages/kghub_downloader/download_utils.py", line 75, in download_from_yaml
15:01:58      download_from_api(item, outfile)
15:01:58    File "/var/lib/jenkins/workspace/add_kghub_downloader_run_jenkins/gitrepo/venv/lib/python3.8/site-packages/kghub_downloader/download_utils.py", line 115, in download_from_api
15:01:58      es_conn = elasticsearch.Elasticsearch(hosts=[yaml_item['url']])
15:01:58    File "/var/lib/jenkins/workspace/add_kghub_downloader_run_jenkins/gitrepo/venv/lib/python3.8/site-packages/elasticsearch/_sync/client/__init__.py", line 312, in __init__
15:01:58      node_configs = client_node_configs(
15:01:58    File "/var/lib/jenkins/workspace/add_kghub_downloader_run_jenkins/gitrepo/venv/lib/python3.8/site-packages/elasticsearch/_sync/client/utils.py", line 101, in client_node_configs
15:01:58      node_configs = hosts_to_node_configs(hosts)
15:01:58    File "/var/lib/jenkins/workspace/add_kghub_downloader_run_jenkins/gitrepo/venv/lib/python3.8/site-packages/elasticsearch/_sync/client/utils.py", line 141, in hosts_to_node_configs
15:01:58      node_configs.append(url_to_node_config(host))
15:01:58    File "/var/lib/jenkins/workspace/add_kghub_downloader_run_jenkins/gitrepo/venv/lib/python3.8/site-packages/elastic_transport/client_utils.py", line 198, in url_to_node_config
15:01:58      raise ValueError(
15:01:58  ValueError: URL must include a 'scheme', 'host', and 'port' component (ie 'https://localhost:9200/')

I think it can be resolved by rolling back elasticsearch to an older version, as this works fine for me locally.