microbiomedata / nmdc-runtime

Runtime system for NMDC data management and orchestration
https://microbiomedata.github.io/nmdc-runtime/
Other
7 stars 3 forks source link

Errors with GOLD ETL #789

Open aclum opened 3 days ago

aclum commented 3 days ago

Describe the bug I tried using dagit-dev to run the GOLD translator code and am getting errors. @sujaypatil96 confirmed that his attempts produce the same error.

To Reproduce Steps to reproduce the behavior:

  1. see error from dagit-dev run here

Screenshots If applicable, add screenshots to help explain your problem.

Acceptance Criteria

Additional context `KeyError: None File "/usr/local/lib/python3.10/site-packages/dagster/_core/execution/plan/utils.py", line 54, in op_execution_error_boundary yield File "/usr/local/lib/python3.10/site-packages/dagster/_utils/init.py", line 482, in iterate_with_context next_output = next(iterator) File "/usr/local/lib/python3.10/site-packages/dagster/_core/execution/plan/compute_generator.py", line 140, in _coerce_op_compute_fn_to_iterator result = invoke_compute_fn( File "/usr/local/lib/python3.10/site-packages/dagster/_core/execution/plan/compute_generator.py", line 128, in invoke_compute_fn return fn(context, args_to_pass) if context_arg_provided else fn(args_to_pass) File "/opt/dagster/lib/nmdc_runtime/site/ops.py", line 661, in nmdc_schema_database_from_gold_study database = translator.get_database() File "/opt/dagster/lib/nmdc_runtime/site/translation/gold_translator.py", line 666, in get_database database.data_generation_set = [ File "/opt/dagster/lib/nmdc_runtime/site/translation/gold_translator.py", line 672, in nmdc_biosample_id=gold_to_nmdc_biosample_ids[