Open mblayman opened 4 days ago
I'm not really a Ruby dev, so this might be an inappropriate solution, but Chat GPT helped me devise this alternative implementation of extract
:
desc "extract", "Extract a single secret from the results of a fetch call"
option :inline, type: :boolean, required: false, hidden: true
def extract(name, secrets)
unescaped_secrets = Shellwords.shellsplit(secrets).first
parsed_secrets = JSON.parse(unescaped_secrets)
...
If fetch
consistently outputs a shell escaped JSON blob in a single line, I think this might work well.
Even with my little hack above, something was not working right. I attempted to continue by switching KAMAL_REGISTRY_PASSWORD
to use an env variable like KAMAL_REGISTRY_PASSWORD=$KAMAL_REGISTRY_PASSWORD
. Then I continued with running kamal setup
. My setup failed for reasons that I think are app-related, but I when I checked on the web.env
file on the server, I found some rather unexpected output. It looks like the command substitution did not happen because I found the following in web.env
:
SECRET_KEY=$(kamal secrets extract SECRET_KEY \\{\\"some-vault/some-item/KAMAL_REGISTRY_PASSWORD\\":\\"the_password_was_here\\",\\"some-vault/some-item/SECRET_KEY\\":\\"the_key_was_here\\"\\})
I confirmed in my Django shell that the SECRET_KEY setting was set to that long raw value. My best guess is that not doing the command substitution was also why my KAMAL_REGISTRY_PASSWORD
did not work until I changed it to use an env variable.
For completeness, here is my .kamal/secrets
(with the sensitive bits removed):
# Secrets defined here are available for reference under registry/password, env/secret, builder/secrets,
# and accessories/*/env/secret in config/deploy.yml. All secrets should be pulled from either
# password manager, ENV, or a file. DO NOT ENTER RAW CREDENTIALS HERE! This file needs to be safe for git.
SECRETS=$(kamal secrets fetch --adapter 1password --account *** --from some-vault/some-item KAMAL_REGISTRY_PASSWORD SECRET_KEY)
#KAMAL_REGISTRY_PASSWORD=$(kamal secrets extract KAMAL_REGISTRY_PASSWORD $SECRETS)
KAMAL_REGISTRY_PASSWORD=$KAMAL_REGISTRY_PASSWORD
SECRET_KEY=$(kamal secrets extract SECRET_KEY $SECRETS)
Thanks for the report @mblayman!
This worked ok when you specified the kamal secrets
commands inside .kamal/secrets
, but that was because it "inlines" the calls to the secret commands and the inlining was unescaping the JSON for us. I've moved things around in https://github.com/basecamp/kamal/pull/1008 so it should work ok now both on the command line and in .kamal/secrets
Ok that approach won't work, the problem is that the dotenv
gem and the shell don't work in the same way.
https://github.com/basecamp/kamal/pull/1009 adds a kamal secrets print
command that can be used for debugging. I'll update the docs to explain this once it is released.
Ran into the same issue. FWIW, this is my current workaround in .kamal/secrets
:
KAMAL_REGISTRY_PASSWORD=$(op read op://vault/item/field)
@blvrd, yeah, I ended up doing the same thing and used the 1password CLI directly just like you did.
Hi!
I'm trying to use the new secret helpers from Kamal 2 to manage secrets for my project. I tried to follow the style listed in the secrets documentation under "Environment variables", but I'm running into a problem with shell escaping.
I think what is happening is that the raw secrets JSON data is getting to the Ruby parsing code with the shell escape characters still intact. Parsing fails because the JSON parser doesn't accept the backslashes.
I tried to do some
puts
debugging in theextract
method oflib/kamal/cli/secrets.rb
to confirm what was going into theJSON.parse
call. You can see from my output below that the shell escape characters are still present.I attempted this on bash and zsh to try to rule out differences in shells.
Is there supposed to be a different way to invoke
kamal secrets fetch
orkamal secrets extract
?Thanks for the help! I hope this bug report is useful. Please let me know if y'all need any other info.