DataBiosphere / dsub

Open-source command-line tool to run batch computing tasks and workflows on backend services such as Google Cloud.
Apache License 2.0
265 stars 44 forks source link

dsub exit codes #192

Open slagelwa opened 4 years ago

slagelwa commented 4 years ago

Just wondering what one should expect if they were to run the following:

dsub --provider local --command 'exit 99' --logging /tmp/log

Right now dsub returns an error code of 1. Was wondering how one would get the actual error code of the command or script that was run? Looks like the only way to do this is to look in the status detail for the Docker exit code in the status detail.

mbookman commented 4 years ago

Hi Joe,

Can you describe your goal here? The Pipelines API doesn't give you the error code from the command, so one doesn't typically build logic based on the failure reason. The existence of the local provider is to enable rapid development of code that one will ultimately execute on Cloud.

Just trying to understand what you would do with the exit code? The failure should be logged, including the error code:

  if [[ $code != "0" ]]; then
    write_event "fail"
    write_status "FAILURE"
    log_error "${message} on or near line ${parent_lineno}; ${0} exiting with status ${code}"
  fi

Thanks.

slagelwa commented 4 years ago

Goal -- I didn't really have one for the error codes to be honest but its more just an observation.

What I was doing was putting together a little GCP Cloud run service that uses dsub to launch long running tasks. Its a bit easier to do then using GCP Cloud functions. (And since we use dsub so much, a little more practical and easy way to architect this).

I was writing some unit tests for the service which is why I was using the local provider and which for obvious reasons is perfect for such tests. I thought I'd throw in an error case as a test, hence the simple "exit 99" command. That's when I discovered the error code doesn't get propagated back up.

Frankly I think we all know that your lucky if any bioinformatics software actually even returns an error code let alone any meaningful code. So it isn't really much of a concern, more so just being complete.