can't access workflow logs from AoU RWB

[ruchim]

I'm trying to use Cromshell from the AoU RWB

I ran a workflow and status succeeded (bold)

Then, when I go to get logs, it fails and the stacktrace (see below) isn't very clear to me. Trying to debug how to fix the logs functionality

> jupyter@5cbce7741997:~$ cromshell-alpha status c47be75c-ba47-4b9e-99a8-8e5cf5d70bc4 2023-05-12 16:23:52,910 cromshell.main INFO Invoked by: /opt/conda/bin/cromshell-alpha status c47be75c-ba47-4b9e-99a8-8e5cf5d70bc4

2023-05-12 16:23:52,910 cromshell.main INFO Log level set to: INFO 2023-05-12 16:23:52,910 cromshell.utilities.cromshellconfig INFO Workflow id and cromwell server not specified.Using default cromwell server 2023-05-12 16:23:52,910 cromshell.utilities.cromshellconfig INFO Server: https://notebooks.firecloud.org/proxy/google/v1/apps/terra-vpc-sc-73ac72b9/all-of-us-9852-cromwell-2473/cromwell-service 2023-05-12 16:23:52,911 cromshell.utilities.cromshellconfig INFO Setting requests timeout from value in config file. 2023-05-12 16:23:52,911 cromshell.utilities.cromshellconfig INFO Request Timeout value: 5 sec 2023-05-12 16:23:52,911 cromshell.utilities.cromshellconfig INFO Will send auth header with token for pet-26799372794334fc31363@terra-vpc-sc-73ac72b9.iam.gserviceaccount.com from config. 2023-05-12 16:23:52,911 cromshell.utilities.cromshellconfig INFO Will send referer header https://notebooks.firecloud.org from config. 2023-05-12 16:23:52,912 cromshell.status.command INFO status 2023-05-12 16:23:52,912 cromshell.utilities.cromshellconfig INFO Checking submission file for associated cromwell server with the provided workflow id. 2023-05-12 16:23:52,912 cromshell.utilities.cromshellconfig INFO Cromwell server set to matching workflow id insubmission file. 2023-05-12 16:23:52,912 cromshell.utilities.cromshellconfig INFO WorkflowID: c47be75c-ba47-4b9e-99a8-8e5cf5d70bc4 2023-05-12 16:23:52,912 cromshell.utilities.cromshellconfig INFO Server: https://notebooks.firecloud.org/proxy/google/v1/apps/terra-vpc-sc-73ac72b9/all-of-us-9852-cromwell-2473/cromwell-service

             __
  .,-;-;-,. /'_\
_/_/_/_|_\_\) /

'-<_><_><_><_>=/\ `//====//-'_\ "" "" ""

> {"status":"Succeeded", "id":"c47be75c-ba47-4b9e-99a8-8e5cf5d70bc4"} jupyter@5cbce7741997:~$ cromshell-alpha logs c47be75c-ba47-4b9e-99a8-8e5cf5d70bc4 2023-05-12 16:24:05,081 cromshell.main INFO Invoked by: /opt/conda/bin/cromshell-alpha logs c47be75c-ba47-4b9e-99a8-8e5cf5d70bc4 2023-05-12 16:24:05,082 cromshell.main INFO Log level set to: INFO 2023-05-12 16:24:05,082 cromshell.utilities.cromshellconfig INFO Workflow id and cromwell server not specified.Using default cromwell server 2023-05-12 16:24:05,082 cromshell.utilities.cromshellconfig INFO Server: https://notebooks.firecloud.org/proxy/google/v1/apps/terra-vpc-sc-73ac72b9/all-of-us-9852-cromwell-2473/cromwell-service 2023-05-12 16:24:05,082 cromshell.utilities.cromshellconfig INFO Setting requests timeout from value in config file. 2023-05-12 16:24:05,082 cromshell.utilities.cromshellconfig INFO Request Timeout value: 5 sec 2023-05-12 16:24:05,083 cromshell.utilities.cromshellconfig INFO Will send auth header with token for pet-26799372794334fc31363@terra-vpc-sc-73ac72b9.iam.gserviceaccount.com from config. 2023-05-12 16:24:05,083 cromshell.utilities.cromshellconfig INFO Will send referer header https://notebooks.firecloud.org from config. 2023-05-12 16:24:05,084 cromshell.logs.command INFO logs 2023-05-12 16:24:05,084 cromshell.utilities.cromshellconfig INFO Checking submission file for associated cromwell server with the provided workflow id. 2023-05-12 16:24:05,084 cromshell.utilities.cromshellconfig INFO Cromwell server set to matching workflow id insubmission file. 2023-05-12 16:24:05,084 cromshell.utilities.cromshellconfig INFO WorkflowID: c47be75c-ba47-4b9e-99a8-8e5cf5d70bc4 2023-05-12 16:24:05,084 cromshell.utilities.cromshellconfig INFO Server: https://notebooks.firecloud.org/proxy/google/v1/apps/terra-vpc-sc-73ac72b9/all-of-us-9852-cromwell-2473/cromwell-service 2023-05-12 16:24:06,164 cromshell.logs.command INFO Status keys set to ['Failed'] Traceback (most recent call last): File "/opt/conda/bin/cromshell-alpha", line 8, in sys.exit(main_entry()) File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 1128, in call return self.main(args, kwargs) File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 1053, in main rv = self.invoke(ctx) File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 1659, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 1395, in invoke return ctx.invoke(self.callback, ctx.params) File "/opt/conda/lib/python3.7/site-packages/click/core.py", line 754, in invoke return __callback(args, *kwargs) File "/opt/conda/lib/python3.7/site-packages/click/decorators.py", line 38, in new_func return f(get_current_context().obj, args, **kwargs) File "/opt/conda/lib/python3.7/site-packages/cromshell/logs/command.py", line 75, in main print_logs=print_logs, File "/opt/conda/lib/python3.7/site-packages/cromshell/logs/command.py", line 102, in obtain_and_print_logs verify_certs=config.requests_verify_certs, TypeError: get_workflow_metadata() missing 1 required positional argument: 'headers'

[SHuang-Broad]

@bshifaw not sure if #259 fixes this

[bshifaw]

Hi @ruchim, I see you are running cromshell-alpha, could you try installing the latest version of cromshell on RW. One way to do this is through 'pip install git+https://github.com/broadinstitute/cromshell.gitand try logs again. (the recent release is called viacromshellas opposed tocromshell-alpha`)

[ruchim]

Tried an update (i can confirm i'm using cromshell 2.0.0)! The logging level is so much nicer (yay!) Still same issue with using the logs command tho

jupyter@c1b74825647d:~$ cromshell status test-1

                 __
      .,-;-;-,. /'_\
    _/_/_/_|_\_\) /
  '-<_><_><_><_>=/\
    `/_/====/_/-'\_\
     ""     ""    ""

{"status":"Succeeded",
"id":"3f1287e8-6442-43e5-b8b9-a81daf4a1a33"}
jupyter@c1b74825647d:~$ cromshell logs test-1
No logs with status ['Failed'] found for workflow, try adding the argument '-s ALL' to list logs with any status
jupyter@c1b74825647d:~$ cromshell version
cromshell 2.0.0

[bshifaw]

This looks good. By default, the logs command will retrieve the logs of tasks with the "Failed " status. Since the workflow being checked here completed with the status of "Succeeded", the logs command wont find any failed tasks and will exit with

No logs with status ['Failed'] found for workflow, try adding the argument '-s ALL' to list logs with any status

This default was set because we assume users are mainly looking at the logs of a workflow because they want to investigate a failure.