Closed justaddcoffee closed 3 years ago
@deepakunni3, getting close, but still seeing an error in the merge step:
18:48:57 + . venv/bin/activate
18:48:57 + deactivate nondestructive
18:48:57 + [ -n ]
18:48:57 + [ -n ]
18:48:57 + [ -n -o -n ]
18:48:57 + [ -n ]
18:48:57 + unset VIRTUAL_ENV
18:48:57 + [ ! nondestructive = nondestructive ]
18:48:57 + VIRTUAL_ENV=/var/lib/jenkins/workspace/9_switch_to_main_kgx_run_jenkins/gitrepo/venv
18:48:57 + export VIRTUAL_ENV
18:48:57 + _OLD_VIRTUAL_PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
18:48:57 + PATH=/var/lib/jenkins/workspace/9_switch_to_main_kgx_run_jenkins/gitrepo/venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
18:48:57 + export PATH
18:48:57 + [ -n ]
18:48:57 + [ -z ]
18:48:57 + _OLD_VIRTUAL_PS1=$
18:48:57 + [ x(venv) != x ]
18:48:57 + PS1=(venv) $
18:48:57 + export PS1
18:48:57 + [ -n -o -n ]
18:48:57 + python3.7 run.py merge -y merge_jenkins.yaml
18:48:59 /var/lib/jenkins/workspace/9_switch_to_main_kgx_run_jenkins/gitrepo/venv/lib/python3.7/site-packages/biolinkml/__init__.py:158: UserWarning: Some URL processing will fail with python 3.7.5 or earlier. Current version: sys.version_info(major=3, minor=7, micro=5, releaselevel='final', serial=0)
18:48:59 warn(f"Some URL processing will fail with python 3.7.5 or earlier. Current version: {sys.version_info}")
18:48:59 [KGX][cli_utils.py][ merge] INFO: Spawning process for 'drug-central'
18:48:59 [KGX][cli_utils.py][ merge] INFO: Spawning process for 'pharmgkb'
18:48:59 [KGX][cli_utils.py][ merge] INFO: Spawning process for 'STRING'
18:48:59 [KGX][cli_utils.py][ merge] INFO: Spawning process for 'ttd'
18:48:59 [KGX][cli_utils.py][ merge] INFO: Spawning process for 'zhou-host-proteins'
18:48:59 [KGX][cli_utils.py][ parse_source] INFO: Processing source 'drug-central'
18:48:59 [KGX][cli_utils.py][ merge] INFO: Spawning process for 'SciBite-CORD-19'
18:48:59 [KGX][cli_utils.py][ merge] INFO: Spawning process for 'sars-cov-2-gene-annot'
18:48:59 [KGX][cli_utils.py][ merge] INFO: Spawning process for 'intact'
18:48:59 [KGX][cli_utils.py][ merge] INFO: Spawning process for 'chembl'
18:48:59 [KGX][cli_utils.py][ merge] INFO: Spawning process for 'gene-ontology'
18:48:59 [KGX][cli_utils.py][ merge] INFO: Spawning process for 'mondo-ontology'
18:48:59 [KGX][cli_utils.py][ merge] INFO: Spawning process for 'chebi'
18:48:59 [KGX][cli_utils.py][ merge] INFO: Spawning process for 'hp-ontology'
18:48:59 [KGX][cli_utils.py][ merge] INFO: Spawning process for 'go-cams'
18:48:59 [KGX][cli_utils.py][ parse_source] INFO: Processing source 'pharmgkb'
18:49:01 [KGX][cli_utils.py][ parse_source] INFO: Processing source 'STRING'
19:11:22 [KGX][cli_utils.py][ parse_source] INFO: Processing source 'ttd'
19:11:22 [KGX][cli_utils.py][ parse_source] INFO: Processing source 'zhou-host-proteins'
19:11:22 [KGX][cli_utils.py][ parse_source] INFO: Processing source 'SciBite-CORD-19'
19:24:43 [KGX][cli_utils.py][ parse_source] INFO: Processing source 'sars-cov-2-gene-annot'
19:24:43 [KGX][cli_utils.py][ parse_source] INFO: Processing source 'intact'
19:24:43 [KGX][cli_utils.py][ parse_source] INFO: Processing source 'chembl'
19:24:43 [KGX][cli_utils.py][ parse_source] INFO: Processing source 'gene-ontology'
19:24:53 [KGX][cli_utils.py][ parse_source] INFO: Processing source 'mondo-ontology'
19:24:58 [KGX][cli_utils.py][ parse_source] INFO: Processing source 'chebi'
19:25:20 [KGX][cli_utils.py][ parse_source] INFO: Processing source 'hp-ontology'
19:25:21 [KGX][cli_utils.py][ parse_source] INFO: Processing source 'go-cams'
19:25:22 Traceback (most recent call last):
19:25:22 File "run.py", line 165, in <module>
19:25:22 cli()
19:25:22 File "/var/lib/jenkins/workspace/9_switch_to_main_kgx_run_jenkins/gitrepo/venv/lib/python3.7/site-packages/click/core.py", line 829, in __call__
19:25:22 return self.main(*args, **kwargs)
19:25:22 File "/var/lib/jenkins/workspace/9_switch_to_main_kgx_run_jenkins/gitrepo/venv/lib/python3.7/site-packages/click/core.py", line 782, in main
19:25:22 rv = self.invoke(ctx)
19:25:22 File "/var/lib/jenkins/workspace/9_switch_to_main_kgx_run_jenkins/gitrepo/venv/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
19:25:22 return _process_result(sub_ctx.command.invoke(sub_ctx))
19:25:22 File "/var/lib/jenkins/workspace/9_switch_to_main_kgx_run_jenkins/gitrepo/venv/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
19:25:22 return ctx.invoke(self.callback, **ctx.params)
19:25:22 File "/var/lib/jenkins/workspace/9_switch_to_main_kgx_run_jenkins/gitrepo/venv/lib/python3.7/site-packages/click/core.py", line 610, in invoke
19:25:22 return callback(*args, **kwargs)
19:25:22 File "run.py", line 84, in merge
19:25:22 load_and_merge(yaml, processes)
19:25:22 File "/var/lib/jenkins/workspace/9_switch_to_main_kgx_run_jenkins/gitrepo/kg_covid_19/merge_utils/merge_kg.py", line 36, in load_and_merge
19:25:22 merged_graph = merge(yaml_file, processes=processes)
19:25:22 File "/var/lib/jenkins/workspace/9_switch_to_main_kgx_run_jenkins/gitrepo/venv/lib/python3.7/site-packages/kgx/cli/cli_utils.py", line 409, in merge
19:25:22 graphs = [r.get() for r in results]
19:25:22 File "/var/lib/jenkins/workspace/9_switch_to_main_kgx_run_jenkins/gitrepo/venv/lib/python3.7/site-packages/kgx/cli/cli_utils.py", line 409, in <listcomp>
19:25:22 graphs = [r.get() for r in results]
19:25:22 File "/usr/lib/python3.7/multiprocessing/pool.py", line 657, in get
19:25:22 raise self._value
19:25:22 multiprocessing.pool.MaybeEncodingError: Error sending result: '<networkx.classes.multidigraph.MultiDiGraph object at 0x7f2ef0b85f90>'. Reason: 'error("'i' format requires -2147483648 <= number <= 2147483647")'
Getting closer, but now seeing this error
06:49:42 BRANCH_NAME=switch_to_main_kgx_run_jenkins
06:49:42 + python3.8 run.py transform
06:49:43 Traceback (most recent call last):
06:49:43 File "run.py", line 6, in <module>
06:49:43 from kg_covid_19 import download as kg_download
06:49:43 File "/var/lib/jenkins/workspace/9_switch_to_main_kgx_run_jenkins/gitrepo/kg_covid_19/__init__.py", line 2, in <module>
06:49:43 from .transform import transform
06:49:43 File "/var/lib/jenkins/workspace/9_switch_to_main_kgx_run_jenkins/gitrepo/kg_covid_19/transform.py", line 8, in <module>
06:49:43 from kg_covid_19.transform_utils.gocam_transform.gocam_transform import GocamTransform
06:49:43 File "/var/lib/jenkins/workspace/9_switch_to_main_kgx_run_jenkins/gitrepo/kg_covid_19/transform_utils/gocam_transform/__init__.py", line 1, in <module>
06:49:43 from .gocam_transform import GocamTransform
06:49:43 File "/var/lib/jenkins/workspace/9_switch_to_main_kgx_run_jenkins/gitrepo/kg_covid_19/transform_utils/gocam_transform/gocam_transform.py", line 6, in <module>
06:49:43 from kgx import RdfTransformer, PandasTransformer # type: ignore
06:49:43 File "/var/lib/jenkins/workspace/9_switch_to_main_kgx_run_jenkins/gitrepo/venv/lib/python3.8/site-packages/kgx/__init__.py", line 1, in <module>
06:49:43 from kgx.transformers.pandas_transformer import PandasTransformer
06:49:43 File "/var/lib/jenkins/workspace/9_switch_to_main_kgx_run_jenkins/gitrepo/venv/lib/python3.8/site-packages/kgx/transformers/pandas_transformer.py", line 10, in <module>
06:49:43 from kgx.utils.kgx_utils import generate_edge_key, generate_uuid
06:49:43 File "/var/lib/jenkins/workspace/9_switch_to_main_kgx_run_jenkins/gitrepo/venv/lib/python3.8/site-packages/kgx/utils/kgx_utils.py", line 6, in <module>
06:49:43 from biolinkml.meta import TypeDefinitionName, ElementName, SlotDefinition, ClassDefinition, TypeDefinition, Element
06:49:43 File "/var/lib/jenkins/workspace/9_switch_to_main_kgx_run_jenkins/gitrepo/venv/lib/python3.8/site-packages/biolinkml/meta.py", line 27, in <module>
06:49:43 from includes.annotations import Annotation
06:49:43 File "/var/lib/jenkins/workspace/9_switch_to_main_kgx_run_jenkins/gitrepo/venv/lib/python3.8/site-packages/includes/annotations.py", line 27, in <module>
06:49:43 from includes.extensions import Extension
06:49:43 File "/var/lib/jenkins/workspace/9_switch_to_main_kgx_run_jenkins/gitrepo/venv/lib/python3.8/site-packages/includes/extensions.py", line 47, in <module>
06:49:43 class Extension(YAMLRoot):
06:49:43 File "/usr/lib/python3.8/dataclasses.py", line 1000, in dataclass
06:49:43 return wrap(cls)
06:49:43 File "/usr/lib/python3.8/dataclasses.py", line 992, in wrap
06:49:43 return _process_class(cls, init, repr, eq, order, unsafe_hash, frozen)
06:49:43 File "/usr/lib/python3.8/dataclasses.py", line 909, in _process_class
06:49:43 _init_fn(flds,
06:49:43 TypeError: dataclasses_init_fn_with_kwargs() missing 1 required positional argument: 'globals'
Fixed by #419
Right now we are pinned to Deepak's fork of KGX (see requirements.txt).
It'd be good to switch to the main KGX branch here
Deepak and I made this PR - this should fix this ticket. We are testing this now.