cromshell-beta logs KeyError

[wharvey31]

cromshell-beta logs results in the following error when run on Researcher Workbench.

Traceback (most recent call last):
  File "/home/jupyter/.local/bin/cromshell-beta", line 8, in <module>
    sys.exit(main_entry())
  File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/opt/conda/lib/python3.7/site-packages/click/decorators.py", line 38, in new_func
    return f(get_current_context().obj, *args, **kwargs)
  File "/home/jupyter/.local/lib/python3.7/site-packages/cromshell/logs/command.py", line 78, in main
    print_logs=print_logs,
  File "/home/jupyter/.local/lib/python3.7/site-packages/cromshell/logs/command.py", line 115, in obtain_and_print_logs
    cat_logs=print_logs,
  File "/home/jupyter/.local/lib/python3.7/site-packages/cromshell/logs/command.py", line 142, in print_workflow_logs
    tasks = list(workflow_metadata["calls"].keys())
KeyError: 'calls'

[bshifaw]

I was able to replicate the error with a workflow that failed before it was submitted Cromwell. The metadata for this workflow has no value for the 'calls' key, hence the error "KeyError: 'calls'". It's likely similar to your situation, you can check by running the metadata subcommand on the workflow-id to check the contents of the calls key and check for failures

[bshifaw]

@wharvey31 , a fix was added such that if cromshell comes across workflow metadata from Cromwell with a empty calls key it will print out a more useful error message.

For example:

Traceback (most recent call last):
  File "/Users/bshifaw/work/cromshell2_project/cromshell/venv/bin/cromshell-beta", line 8, in <module>
    sys.exit(main_entry())
  File "/Users/bshifaw/work/cromshell2_project/cromshell/venv/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/Users/bshifaw/work/cromshell2_project/cromshell/venv/lib/python3.9/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/Users/bshifaw/work/cromshell2_project/cromshell/venv/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/bshifaw/work/cromshell2_project/cromshell/venv/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/bshifaw/work/cromshell2_project/cromshell/venv/lib/python3.9/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/Users/bshifaw/work/cromshell2_project/cromshell/venv/lib/python3.9/site-packages/click/decorators.py", line 38, in new_func
    return f(get_current_context().obj, *args, **kwargs)
  File "/Users/bshifaw/work/cromshell2_project/cromshell/src/cromshell/logs/command.py", line 67, in main
    obtain_and_print_logs(
  File "/Users/bshifaw/work/cromshell2_project/cromshell/src/cromshell/logs/command.py", line 140, in obtain_and_print_logs
    check_workflow_for_calls(workflow_status_json)
  File "/Users/bshifaw/work/cromshell2_project/cromshell/src/cromshell/logs/command.py", line 95, in check_workflow_for_calls
    raise KeyError(
KeyError: "Empty 'calls' key found in workflow metadata. Workflow failed with the following error(s): [{'causedBy': [{'message': 'Task HelloWorldTask has an invalid runtime attribute docker = !! NOT FOUND !!', 'causedBy': []}], 'message': 'Runtime validation failed'}]"

The example above is one where the calls key is empty and the metadata contained a failure message. If there isn't a failure message attached to the workflow metadata then a general message will be printed:

"KeyError:  Empty 'calls' key found in workflow metadata. This may indicate no tasks were run by the workflow, 
the workflow has yet to run any tasks, or a failure occurred before the workflow started."

[SHuang-Broad]

@wharvey31 a fix has been just merged into the cromshell2.0 branch. Please let us know if you still run into this problem.