Hephy builder fails when app uses AWS reserved environment variables

[eudalov]

Hephy Builder fails to store the Docker image it has just built with the message below:

Writing objects: 100% (4/4), 356 bytes | 0 bytes/s, done.
Total 4 (delta 3), reused 0 (delta 0)
Starting build... but first, coffee\!
2020/09/29 15:22:49 InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.
remote: status code: 403, request id: 579ED5F7E401E3FC, host id: R+lwtdSuUO8clv3fDGrKW0V9IR8ZjnpeD17zcZ/1x5nF8ny9gDK4JIxTT3NPmB2vrfFy9b3tCwo=
Traceback (most recent call last):
  File "/deploy.py", line 89, in <module>
    download_file(tar_path)
  File "/deploy.py", line 83, in download_file
    subprocess.check_call(command)
  File "/usr/lib/python3.5/subprocess.py", line 581, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '\['objstorage', '--storage-type=s3', 'download', 'home/ilias-custom-metrics-app:git-0f96a9da/tar', 'apptar']' returned non-zero exit status 1
remote: 2020/09/29 15:22:55 Error running git receive hook \[Build pod exited with code 1, stopping build.]

Steps to reproduce:

• Pick up any codebase with a Dockerfile.
• Create an application on EYK for this codebase
• For that application, set env vars AWS_ACCESS_KEY_ID, AWS_BUCKET_NAME, AWS_SECRET_ACCESS_KEY to any valid value that is not the credentials for the account where the cluster is running.
• Run git push deis

[dvalfre]

The presence of the AWS_ vars on the running environment prompts dockerbuilder to use them instead of using the ones configured when installed. The impact is that dockerbuilder will potentially attempt to store the resulting artifacts on a different AWS account than the one configured at installation time.

Our research found two ways of addressing this: a) Modify dockerbuilder so that it sanitizes the relevant env vars before pushing to S3 the resulting artifact, by inserting a step before downloading the tar file b) Modify object-storage-cli to have it ignore AWS related variables on the environment.

Having said all of that, these fixes/approaches do not consider this case: when the user intentionally wants to override the default credentials and 'force' dockerbuilder to push the artifact to a different AWS account. If any member of the community is using this scenario then we won't be able to submit the fix, and will just share it as a snippet.

[kingdonb]

I can't imagine a scenario myself where it's considered correct behavior for the builder to change its push target for artifacts based on the application's configuration, and I'd have to see an example of how it could be used productively to even consider that as intentional. I am for fixing the bug, so apps with AWS_S3 storage buckets of their own to attach can do so using environment variables and not interfere with Builder or its work. 👍

[Cryptophobia]

This is quite a serious bug. I'm surprised I have not run into it more often. Maybe because a lot of the apps we deployed on hephy were dockerized containers.

[Cryptophobia]

PR #14 merged! Thank you for the contribution @eudalov and @dvalfre