Loading destination-specific secrets in deploy.yml

[nickhammond]

The docs mention that you can load dotenv if you need to reference those secrets in deploy.yml. https://kamal-deploy.org/docs/upgrading/secrets-changes/#environment-variables-in-deployyml

If you use destinations though, there's no way to dynamically load those secrets because ENV['KAMAL_DESTINATION'] is empty.

.kamal/secrets

RAILS_ENV=$RAILS_ENV

.kamal/secrets.staging (not committed to git)

RAILS_ENV=staging

config/deploy.yml

<% 
require "dotenv"
puts "Loaded from deploy.yml"
puts env: ENV['RAILS_ENV']
puts destination: ENV['KAMAL_DESTINATION']
Dotenv.load(".kamal/secrets", overwrite: true)
puts env: ENV['RAILS_ENV']
puts destination: ENV['KAMAL_DESTINATION']
%>

config/deploy.staging.yml

<% 
require "dotenv"
puts "Loaded from staging"
%>

$ kamal build deliver -d staging
Loaded from deploy.yml
{:env=>nil}
{:destination=>nil}
{:env=>""}
{:destination=>nil}

I also just tried loading Dotenv.load(".kamal/secrets.staging", overwrite: true) in config/deploy.staging.yml but that code doesn't actually run.

I was thinking it would be possible to just add this at the top of config/deploy.yml and secrets would get loaded into Kamal.

Dotenv.load(".kamal/secrets.#{ENV['KAMAL_DESTINATION']}", overwrite: true)

Another way to look at it would be, how do you reference the destination so you can reference different environment/destination secrets for config/deploy.yml?

SECRETS=$(kamal secrets fetch ...)

[djmb]

I also just tried loading Dotenv.load(".kamal/secrets.staging", overwrite: true) in config/deploy.staging.yml but that code doesn't actually run.

I would expect this to run - is the problem that deploy.yml is parsed first so the secrets are not loaded there?

We could add the destination I guess, but the secrets should generally be referenced just by name and substituted by Kamal when required.

And if they are not actually secrets but just regular environment variables, then can just be referenced directly in the relevant deploy.yml/deploy.destination.yml files.

Would one of those options work, or are you doing something where that's a bit inconvenient?

[nickhammond]

I would expect this to run - is the problem that deploy.yml is parsed first so the secrets are not loaded there?

Yes, this looks to be the issue because deploy.yml is loaded first which is referencing things that are attempting to get set in deploy.staging.yml.

And if they are not actually secrets but just regular environment variables, then can just be referenced directly in the relevant deploy.yml/deploy.destination.yml files.

If RAILS_ENV is set in deploy.staging.yml, how do I reference that value when it's not in a env/secret section? I'm guessing that's the issue is that I'm utilizing ENV in unexpected places.

Here's a few examples of how I'm using those values. I like to write the destination-specific recipes so that they only contain information that you want to override in the default recipe. Similar to Rails, similar to the old Capistrano style.

I typically have a simple inventory file which contains a references to the various role IPs for a service, worker and web for example.

servers:
  web:
    hosts:
      - <%= JSON.parse(File.read("terraform/#{ENV['RAILS_ENV']}.json")).dig('web_ip_address', 'value') %>

For this example I could of course copy this over or move this to the destination-specific recipe file. For more complicated apps where there's a few web services, a few workers, it's not as manageable to keep up with those changes between the destinations.

For example, say you have 4 worker roles and the only difference is the worker count in each environment. Staging only needs a few working on a queue, production might need significantly more. Loading a config file that already has concurrency set for that environment is really helpful instead of having to bump concurrency in two places(sidekiq config, kamal config).

  worker_default:
    hosts:
      - <%= JSON.parse(File.read("terraform/#{ENV['RAILS_ENV']}.json")).dig('worker_default_ip_addresses', 'value') %>
    cmd: bundle exec sidekiq -g default -C config/sidekiq.yml
    env:
      clear:
        WORKER: true
        DB_POOL: <%= YAML.load(File.read("config/sidekiq.yml"))[ENV['RAILS_ENV']][:concurrency] + 5 %>
    healthcheck:
      cmd: bin/sidekiq-healthcheck

And then for a small hobby app I'm hosting staging and production on the same machine, so I'm setting the MySQL storage directory from that value.

    directories:
      - mysql-<%= ENV['RAILS_ENV'] %>:/var/lib/mysql

These are all being used currently with Kamal 1.9 btw. After writing out all of these examples too it looks like it's really just RAILS_ENV or KAMAL_DESTINATION that's needed or changing behavior. Am I missing something though or is there a way to do this currently?

[djmb]

Thanks @nickhammond - that's all makes sense, and I think making KAMAL_DESTINATION available makes sense 👍

[nickhammond]

@djmb Great, opened a PR! #1019. I'll close this and we can move things over there.