Add support for cromwell behind authorization layer

[tlangs]

Cromshell is currently useless for use against a cromwell server that requires authentication. Adding an ability to use a bearer token, by either passing it in as input specifically or using some env variable, would make allow us to use Cromshell against more of our servers.

[lbergelson]

@tlangs That sounds like a really good idea but I have some questions about it.

1. How is the token passed to cromwell? As a parameter in each call to the cromwell server?
2. Do you generally use the same token for every cromwell server you would want to connect to, or do you need a per server configuration?

[tlangs]

For curl the token is passed as a header: curl -H "Authorization: Bearer $ACCESS_TOKEN" $CROMWELL_URL. Whether the same token is used for different servers depends. Right now, our use case is that we'd just use the same token for all the servers because we have uber-access to everything. In the future, there's definitely a use case where a production server should only be accessible through a production service-account, so impersonating that user and using that token for that specific server would be necessary.

My thoughts are to, instead of relying on an env variable for config, writing a config file to ~/.cromshell/ and storing configurations there. That could make it easy to, say, store dev, staging, and prod configurations and be able to reference them by name for an invocation instead of having to reset the env variable every time a different server is needed. The config could also store per-server info like the access token, the default API version, etc...

[jonn-smith]

I like config stores much better than env vars in general. In some ways they're more obvious and accessible.

This shouldn't be too difficult, but might be time consuming as we don't have a good way to test right now. I don't know of any servers that I have access to which require auth tokens (and that I can test on).

My proposed fix for this:

Add a token command to cromshell:

cromshell token add SERVER TOKEN
cromshell token remove [SERVER]

Tokens would be stored in ~/cromshell/access_tokens.tsv in the format: SERVER TOKEN

@lbergelson - thoughts?

[tlangs]

If you need someone to test on an authed server, I can do it.

[jonn-smith]

Cool. I'll double-check that we don't have a machine I can ping, but if we don't I'll reach out when the feature is implemented.

[lbergelson]

That seems generally like a good solution. I might expand it to be more general though.

cromshell server add ALIAS SERVER_URL [TOKEN]
cromshell server list

[dvoet]

Access tokens are short lived, usually 1 hour tops, so storing them does not make a ton of sense.

Usually the quick and dirty thing we do is run gcloud auth print-access-token to get an access token for the user currently logged in to gcloud. A more sophisticated approach would be to do an oauth flow and store a refresh token locally (similar to what cloud auth login does) but then you need to figure out how to distribute a client secret and the google project setup for oauth would need to be whitelisted with firecloud.

[jonn-smith]

gcloud auth print-access-token is a fine solution - we can run it before talking to the server when we build the request and fail if it fails.

All this would mean is that the stored auth data goes from a token to a boolean (does it need auth or not), which is easier in some ways.

[aednichols]

gcloud auth print-access-token is a fine solution - we can run it before talking to the server when we build the request and fail if it fails.

For Cromshell 2.0, implemented in https://github.com/broadinstitute/cromshell/pull/208