Additional fixes for graph build

caufieldjh commented 1 year ago

[x] Make Jenkins calls to python explicit for Py3.9
[x] Update build to use new, fresh, project-specific Docker image
[x] Handle various issues in DrugCentral references ingest, including mapping journals to NLM Catalog IDs
[x] Repair DrugCentral property ingest
[x] Ensure all ingests specify categories
[x] Adjust compressed graph file creation in Jenkinsfile so result contains no relative paths
[x] Repair Orphanet ingest (gene to disease and phenotype to disease)
[x] Repair OMIM ingest
[x] Repair TCRD ingest (doesn't ingest full DB right now)

caufieldjh commented 1 year ago

Transform appears to work as expected until it gets to parsing the DrugCenral dump:

19:13:25  ERROR:root:Encountered error: [Errno 2] No such file or directory: 'data/raw/drugcentral.dump.010_05_2021.sql.gz'

caufieldjh commented 1 year ago

DrugCentral is parsed properly now, with the exception of a breaking error that is probably (since Jenkins provides output out of order) in the atc_ddd transform:

11:36:00  WARNING:koza.model.config.source_config:Could not load dataset description from metadata file
11:36:00  ERROR:root:Encountered error:  Unknown argument: relation = 'RO:0002351'
11:36:00  Parsing data/raw/mondo_kgx_tsv.tar.gz
11:36:00  Parsing data/raw/chebi_kgx_tsv.tar.gz
11:36:00  Parsing data/raw/hp_kgx_tsv.tar.gz
11:36:00  Parsing data/raw/go_kgx_tsv.tar.gz
11:36:00  Parsing data/raw/ogms_kgx_tsv.tar.gz
11:36:00  Parsing data/raw/drug.target.interaction.tsv.gz
11:36:00  Transforming to data/transformed/drug_central using source in kg_idg/transform_utils/drug_central/drugcentral-dti.yaml
11:36:00  koza_apps entry created for: drugcentral-dti
11:36:00  koza_app: <koza.app.KozaApp object at 0x7f10043ea1f0>
11:36:00  No value found for Ki of ketanserin vs Alpha-2A adrenergic receptor
11:36:00  Parsing data/raw/drugcentral.dump.sql.gz
11:36:00  Transforming data dump to TSV. This may take a while...
11:36:00  Database type: postgres
11:36:00  PostgreSQL server version: 120012
11:36:00  Creating database with name temporary_database
11:36:00  Creating login for user jenkinsuser
11:36:00  Created temporary_database.
11:36:00  Loading data/raw/drugcentral.dump.sql into temporary_database...
11:36:00  Exporting atc_ddd from temporary_database to data/transformed/drug_central/drugcentral-atc_ddd.tsv...
11:36:00  Exporting approval from temporary_database to data/transformed/drug_central/drugcentral-approval.tsv...
11:36:00  Exporting reference from temporary_database to data/transformed/drug_central/drugcentral-reference.tsv...
11:36:00  Exporting property from temporary_database to data/transformed/drug_central/drugcentral-property.tsv...
11:36:00  Exporting identifier from temporary_database to data/transformed/drug_central/drugcentral-identifier.tsv...
11:36:00  Exporting structures from temporary_database to data/transformed/drug_central/drugcentral-structures.tsv...
11:36:00  Complete.
11:36:00  Transforming to data/transformed/drug_central using source in kg_idg/transform_utils/drug_central/drugcentral-atc_ddd.yaml
11:36:00  koza_apps entry created for: drugcentral-atc_ddd
11:36:00  koza_app: <koza.app.KozaApp object at 0x7f100e7d3310>

This may also fix #105

caufieldjh commented 1 year ago

OK, here's a new one:

16:05:59  ERROR:root:Encountered error: J Neurosci is not a valid URI or CURIE

This is probably from drugcentral-reference, since that ingests journal articles and metadata.

caufieldjh commented 1 year ago

The issue may be due to how Biolink expects Article objects to be defined. The published_in slot now has a range of Uriorcurie

caufieldjh commented 1 year ago

Still doesn't quite work - most/all references are invalid and Koza still throws an error about a blank IRI/CURIE:

ERROR:root:Encountered error:  is not a valid URI or CURIE

caufieldjh commented 1 year ago

Or rather, after the last commit the only error is

ERROR:root:Encountered error: ''

caufieldjh commented 1 year ago

In transform.py, every exception is caught: https://github.com/Knowledge-Graph-Hub/kg-idg/blob/a83caac15bd25af2c04891bceec77137bd2dae8c/kg_idg/transform.py#L63-L65

But this isn't very informative when the error text is ''.

Will try disabling that to see what's throwing errors outside the Koza transforms themselves. My suspicion is that it's in the write_reference_map function of the primary DrugCentral transform.

caufieldjh commented 1 year ago

A couple reference issues remain:

Years don't parse as dates. Ex:

17:39:27  {'id': '55', 'pmid': '20590641', 'doi': '10.1111/j.1476-5381.2010.00816.x', 'document_id': '', 'type': 'JOURNAL ARTICLE', 'authors': '', 'title': 'Molecular determinants of state-dependent block of voltage-gated sodium channels by pilsicainide.', 'isbn10': '', 'url': '', 'journal': 'Br J Pharmacol', 'volume': '', 'issue': '', 'dp_year': '2010', 'pages': ''}
17:39:27  Invalid reference: 55: 2010 is not a valid date

Use ISBN as ID if needed, ex:

17:39:27  {'id': '1', 'pmid': '', 'doi': '', 'document_id': '', 'type': 'BOOK', 'authors': 'Karen Baxter', 'title': "Stockley's Drug Interactions", 'isbn10': '0853699143', 'url': '', 'journal': '', 'volume': '', 'issue': '', 'dp_year': '2010', 'pages': ''}
17:39:27  Invalid reference: 1: No valid identifiers found!

Also:

drugcentral-property ingest fails with ValueError: is not a valid URI or CURIE. This could be due to its dependence on the reference map, so if that has any blank IDs, the property ingest will fail like this. It could also be a bug unique to this ingest. Fun fun fun.

caufieldjh commented 1 year ago

The error on each reference is now category must be supplied. Thought this was assigned by the Koza transform since the object type is defined.

But this is just the class directly from the Biolink Model, interpreted by linkml-runtime. It does need category to be provided, so other ingests need that detail, too.

caufieldjh commented 1 year ago

Orphanet ingest is breaking:

16:02:54  Parsing data/raw/orphanet.nt
16:02:54  [KGX][rdf_source.py][              triple] WARNING: Skipping http://purl.org/oban/association_has_object http://www.w3.org/1999/02/22-rdf-syntax-ns#type http://www.w3.org/2002/07/owl#ObjectProperty
16:02:54  [KGX][rdf_source.py][              triple] WARNING: Skipping http://purl.org/oban/association_has_predicate http://www.w3.org/1999/02/22-rdf-syntax-ns#type http://www.w3.org/2002/07/owl#ObjectProperty
16:02:54  [KGX][rdf_source.py][              triple] WARNING: Skipping http://purl.org/oban/association_has_subject http://www.w3.org/1999/02/22-rdf-syntax-ns#type http://www.w3.org/2002/07/owl#ObjectProperty
16:03:04  Traceback (most recent call last):
16:03:04    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/run.py", line 167, in <module>
16:03:04      cli()
16:03:04    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/venv/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
16:03:04      return self.main(*args, **kwargs)
16:03:04    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/venv/lib/python3.9/site-packages/click/core.py", line 1055, in main
16:03:04      rv = self.invoke(ctx)
16:03:04    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/venv/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
16:03:04      return _process_result(sub_ctx.command.invoke(sub_ctx))
16:03:04    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/venv/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
16:03:04      return ctx.invoke(self.callback, **ctx.params)
16:03:04    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/venv/lib/python3.9/site-packages/click/core.py", line 760, in invoke
16:03:04      return __callback(*args, **kwargs)
16:03:04    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/run.py", line 66, in transform
16:03:04      kg_transform(*args, **kwargs)
16:03:04    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/kg_idg/transform.py", line 60, in transform
16:03:04      t.run()
16:03:04    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/kg_idg/transform_utils/orphanet/orphanet.py", line 38, in run
16:03:04      self.parse("orphanet", data_file, ORPHANET_NT_FILENAME)
16:03:04    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/kg_idg/transform_utils/orphanet/orphanet.py", line 51, in parse
16:03:04      transform(
16:03:04    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/venv/lib/python3.9/site-packages/kgx/cli/cli_utils.py", line 560, in transform
16:03:04      transform_source(
16:03:04    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/venv/lib/python3.9/site-packages/kgx/cli/cli_utils.py", line 869, in transform_source
16:03:04      transformer.transform(input_args, output_args)
16:03:04    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/venv/lib/python3.9/site-packages/kgx/transformer.py", line 236, in transform
16:03:04      self.process(source_generator, intermediate_sink)
16:03:04    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/venv/lib/python3.9/site-packages/kgx/transformer.py", line 332, in process
16:03:04      for rec in source:
16:03:04    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/venv/lib/python3.9/site-packages/kgx/source/rdf_source.py", line 167, in parse
16:03:04      yield from p.parse(open(filename, "rb"))
16:03:04    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/venv/lib/python3.9/site-packages/kgx/parsers/ntriples_parser.py", line 47, in parse
16:03:04      yield from self.parseline()
16:03:04    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/venv/lib/python3.9/site-packages/kgx/source/rdf_source.py", line 228, in triple
16:03:04      (element_uri, canonical_uri, predicate, property_name) = self.process_predicate(
16:03:04    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/venv/lib/python3.9/site-packages/kgx/source/rdf_source.py", line 591, in process_predicate
16:03:04      element = self.get_biolink_element(p)
16:03:04    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/venv/lib/python3.9/site-packages/kgx/source/rdf_source.py", line 847, in get_biolink_element
16:03:04      mapping = toolkit.get_element_by_mapping(predicate)
16:03:04    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/venv/lib/python3.9/site-packages/bmt/toolkit.py", line 1114, in get_element_by_mapping
16:03:04      lambda s, l: s.intersection(set(l)), without_empty_lists[1:], set(without_empty_lists[0])
16:03:04  IndexError: list index out of range

caufieldjh commented 1 year ago

I don't have an immediate lead on how to fix that Orphanet ingest so it's currently disabled.

caufieldjh commented 1 year ago

OMIMTransform has the same issue as Orphanet:

17:41:57  [2022-11-04 14:41:51][INFO   ][root    ] Parsing OMIMTransform
17:42:00  Parsing data/raw/omim.nt
17:42:00  [2022-11-04 14:41:59][INFO   ][root    ] Loading schema https://w3id.org/linkml/types from https://raw.githubusercontent.com/biolink/biolink-model/v3.0.3/biolink-model.yaml
17:42:22  Traceback (most recent call last):
17:42:22    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/run.py", line 167, in <module>
17:42:22      cli()
17:42:22    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/venv/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
17:42:22      return self.main(*args, **kwargs)
17:42:22    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/venv/lib/python3.9/site-packages/click/core.py", line 1055, in main
17:42:22      rv = self.invoke(ctx)
17:42:22    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/venv/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
17:42:22      return _process_result(sub_ctx.command.invoke(sub_ctx))
17:42:22    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/venv/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
17:42:22      return ctx.invoke(self.callback, **ctx.params)
17:42:22    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/venv/lib/python3.9/site-packages/click/core.py", line 760, in invoke
17:42:22      return __callback(*args, **kwargs)
17:42:22    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/run.py", line 66, in transform
17:42:22      kg_transform(*args, **kwargs)
17:42:22    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/kg_idg/transform.py", line 60, in transform
17:42:22      t.run()
17:42:22    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/kg_idg/transform_utils/omim/omim.py", line 38, in run
17:42:22      self.parse("omim", data_file, OMIM_NT_FILENAME)
17:42:22    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/kg_idg/transform_utils/omim/omim.py", line 51, in parse
17:42:22      transform(
17:42:22    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/venv/lib/python3.9/site-packages/kgx/cli/cli_utils.py", line 560, in transform
17:42:22      transform_source(
17:42:22    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/venv/lib/python3.9/site-packages/kgx/cli/cli_utils.py", line 869, in transform_source
17:42:22      transformer.transform(input_args, output_args)
17:42:22    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/venv/lib/python3.9/site-packages/kgx/transformer.py", line 236, in transform
17:42:22      self.process(source_generator, intermediate_sink)
17:42:22    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/venv/lib/python3.9/site-packages/kgx/transformer.py", line 332, in process
17:42:22      for rec in source:
17:42:22    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/venv/lib/python3.9/site-packages/kgx/source/rdf_source.py", line 167, in parse
17:42:22      yield from p.parse(open(filename, "rb"))
17:42:22    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/venv/lib/python3.9/site-packages/kgx/parsers/ntriples_parser.py", line 47, in parse
17:42:22      yield from self.parseline()
17:42:22    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/venv/lib/python3.9/site-packages/kgx/source/rdf_source.py", line 228, in triple
17:42:22      (element_uri, canonical_uri, predicate, property_name) = self.process_predicate(
17:42:22    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/venv/lib/python3.9/site-packages/kgx/source/rdf_source.py", line 591, in process_predicate
17:42:22      element = self.get_biolink_element(p)
17:42:22    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/venv/lib/python3.9/site-packages/kgx/source/rdf_source.py", line 847, in get_biolink_element
17:42:22      mapping = toolkit.get_element_by_mapping(predicate)
17:42:22    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/venv/lib/python3.9/site-packages/bmt/toolkit.py", line 1114, in get_element_by_mapping
17:42:22      lambda s, l: s.intersection(set(l)), without_empty_lists[1:], set(without_empty_lists[0])
17:42:22  IndexError: list index out of range

caufieldjh commented 1 year ago

Will temporarily disable OMIM ingest, but plan to come back to it in this PR

caufieldjh commented 1 year ago

Re-adding the Orphanet data should be possible with something like https://github.com/kevinschaper/orphanet-koza-transform

caufieldjh commented 1 year ago

Without OMIM and Orphanet, this completes transforms successfully - or at least it claims to. The merge fails because it can't find the TCRD protein node file:

FileNotFoundError: Filename '/data/transformed/tcrd/tcrd-protein_nodes.tsv' for source 'tcrd-protein' does not exist!

Sure enough, here's what happens during that TCRD ingest:

[2022-11-05T17:30:35.241Z] [2022-11-05 10:30:26][INFO   ][root    ] Parsing TCRDTransform
[2022-11-05T17:30:35.241Z] Parsing data/raw/TCRDv6.12.4.tsv
[2022-11-05T17:30:35.241Z] Transforming to data/transformed/tcrd using source in kg_idg/transform_utils/tcrd/tcrd-ids.yaml
[2022-11-05T17:30:35.241Z] [2022-11-05 10:30:26][INFO   ][koza.app] Transforming source: tcrd-ids
[2022-11-05T17:30:38.469Z] [2022-11-05 10:30:38][INFO   ][koza.io.reader.csv_reader] Finished processing 20412 rows for tcrd-ids from file data/raw/TCRDv6.12.4.tsv
[2022-11-05T17:30:38.469Z] [2022-11-05 10:30:38][INFO   ][koza.io.reader.csv_reader] Finished processing 20412 rows for tcrd-ids from file data/raw/TCRDv6.12.4.tsv
[2022-11-05T17:39:59.724Z] mysql: [Warning] Using a password on the command line interface can be insecure.
[2022-11-05T17:39:59.724Z] ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (13)
[2022-11-05T17:39:59.724Z] mysql: [Warning] Using a password on the command line interface can be insecure.
[2022-11-05T17:39:59.724Z] ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (13)
[2022-11-05T17:39:59.724Z] mysql: [Warning] Using a password on the command line interface can be insecure.
[2022-11-05T17:39:59.724Z] ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (13)
[2022-11-05T17:39:59.724Z] mysql: [Warning] Using a password on the command line interface can be insecure.
[2022-11-05T17:39:59.724Z] ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (13)
[2022-11-05T17:39:59.724Z] Parsing data/raw/tcrd.sql.gz
[2022-11-05T17:39:59.724Z] Transforming MySQL dump to TSV. This may take a while...
[2022-11-05T17:39:59.724Z] Database type: mysql
[2022-11-05T17:39:59.724Z] MySQL server version: 8.0.31-0ubuntu0.20.04.1
[2022-11-05T17:39:59.724Z] Created temporary_database.
[2022-11-05T17:39:59.724Z] Reading data_type from data/raw/tcrd.sql...
[2022-11-05T17:39:59.724Z] Reading info_type from data/raw/tcrd.sql...
[2022-11-05T17:39:59.724Z] Reading xref_type from data/raw/tcrd.sql...
[2022-11-05T17:39:59.724Z] Reading protein from data/raw/tcrd.sql...
[2022-11-05T17:39:59.724Z] Loading data/raw/tcrd-data_type.sql into temporary_database...
[2022-11-05T17:39:59.724Z] Loading data/raw/tcrd-info_type.sql into temporary_database...
[2022-11-05T17:39:59.724Z] Loading data/raw/tcrd-xref_type.sql into temporary_database...
[2022-11-05T17:39:59.724Z] Loading data/raw/tcrd-protein.sql into temporary_database...
[2022-11-05T17:39:59.724Z] Database contains:
[2022-11-05T17:39:59.724Z] Database is empty - please check input file.
[2022-11-05T17:39:59.724Z] Did not process TCRD MySQL dump!

It looks like, despite the connection errors, the function is eventually able to create the temporary database, but nothing gets written to it. Or, it doesn't actually create it. Or it doesn't write to it. Or it writes but can't read. Or something else happens independent of the database.

caufieldjh commented 1 year ago

See also #113 - don't want to put it in this PR, though

caufieldjh commented 1 year ago

Going to deactivate the more complex TCRD ingest for now and just ingest the id file (containing Uniprot IDs, xrefs, and TDLs).

caufieldjh commented 1 year ago

Now there's... a new error.

15:01:10  Parsing data/raw/drug.target.interaction.tsv.gz
15:01:10  Transforming to data/transformed/drug_central using source in kg_idg/transform_utils/drug_central/drugcentral-dti.yaml
15:01:10  [2022-11-08 12:01:05][INFO   ][koza.app] Transforming source: drugcentral-dti
15:01:10  Traceback (most recent call last):
15:01:10    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/run.py", line 167, in <module>
15:01:10      cli()
15:01:10    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/venv/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
15:01:10      return self.main(*args, **kwargs)
15:01:10    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/venv/lib/python3.9/site-packages/click/core.py", line 1055, in main
15:01:10      rv = self.invoke(ctx)
15:01:10    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/venv/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
15:01:10      return _process_result(sub_ctx.command.invoke(sub_ctx))
15:01:10    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/venv/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
15:01:10      return ctx.invoke(self.callback, **ctx.params)
15:01:10    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/venv/lib/python3.9/site-packages/click/core.py", line 760, in invoke
15:01:10      return __callback(*args, **kwargs)
15:01:10    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/run.py", line 66, in transform
15:01:10      kg_transform(*args, **kwargs)
15:01:10    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/kg_idg/transform.py", line 60, in transform
15:01:10      t.run()
15:01:10    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/kg_idg/transform_utils/drug_central/drug_central.py", line 60, in run
15:01:10      self.parse(name, data_file, k)
15:01:10    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/kg_idg/transform_utils/drug_central/drug_central.py", line 188, in parse
15:01:10      transform_source(
15:01:10    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/venv/lib/python3.9/site-packages/koza/cli_runner.py", line 83, in transform_source
15:01:10      source_koza.process_sources()
15:01:10    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/venv/lib/python3.9/site-packages/koza/app.py", line 105, in process_sources
15:01:10      transform_module = importlib.import_module(transform_code)
15:01:10    File "/usr/lib/python3.9/importlib/__init__.py", line 127, in import_module
15:01:10      return _bootstrap._gcd_import(name[level:], package, level)
15:01:10    File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
15:01:10    File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
15:01:10    File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
15:01:10    File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
15:01:10    File "<frozen importlib._bootstrap_external>", line 855, in exec_module
15:01:10    File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
15:01:10    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/kg_idg/transform_utils/drug_central/drugcentral-dti.py", line 50, in <module>
15:01:10      drug = Drug(
15:01:10    File "<string>", line 21, in __init__
15:01:10    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/venv/lib/python3.9/site-packages/biolink/model.py", line 3480, in __post_init__
15:01:10      super().__post_init__(**kwargs)
15:01:10    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/venv/lib/python3.9/site-packages/biolink/model.py", line 3240, in __post_init__
15:01:10      super().__post_init__(**kwargs)
15:01:10    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/venv/lib/python3.9/site-packages/biolink/model.py", line 3180, in __post_init__
15:01:10      super().__post_init__(**kwargs)
15:01:10    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/venv/lib/python3.9/site-packages/biolink/model.py", line 3084, in __post_init__
15:01:10      super().__post_init__(**kwargs)
15:01:10    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/venv/lib/python3.9/site-packages/biolink/model.py", line 1650, in __post_init__
15:01:10      super().__post_init__(**kwargs)
15:01:10    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/venv/lib/python3.9/site-packages/biolink/model.py", line 1610, in __post_init__
15:01:10      super().__post_init__(**kwargs)
15:01:10    File "/var/lib/jenkins/workspace/idg_more_build_fixes_run_jenkins/gitrepo/venv/lib/python3.9/site-packages/linkml_runtime/utils/yamlutils.py", line 48, in __post_init__
15:01:10      raise ValueError('\n'.join(messages))
15:01:10  ValueError:  Unknown argument: source = 'DrugCentral'

caufieldjh commented 1 year ago

I think that's due to a new version of koza.

caufieldjh commented 1 year ago

Last build cleared everything but the post-processing to generate test graphs, so once unit tests / linters clear this should be ready for prime time.

Knowledge-Graph-Hub / kg-idg

Additional fixes for graph build #110