Environment Variables in *.yml files are not resolved

[girishgl]

I am not able to read environment variables from yml file while starting shoryuken worker

bundle exec shoryuken --daemon --pidfile '/tmp/pids/shoryuken.pid' --logfile '/app/log/application.log' --config "/app/config/shoryuken.$RAILS_ENV.yml" --rails

shoryuken.production.yml

aws:
  region: us-east-1
  ssl_verify_peer: false
concurrency: 64
delay: 5
queues:
  - <%= ENV["XYZ_QUEUE"] %>
  - <%= ENV["ABC_QUEUE"] %>

These environments variables are set in config/application.rb

# load environment variables from .env files
require 'dotenv'
Dotenv.load

Dotenv.load('.env.test') if ENV['RAILS_ENV'] == 'test'

require 'aws-sdk-secretsmanager'
client = Aws::SecretsManager::Client.new(region: ENV['AWS_SECRET_MANAGER_REGION']  )

secrets_path = "xyz/#{ENV['RAILS_ENV']}".downcase
secrets = client.get_secret_value(secret_id: secrets_path).secret_string
JSON.parse(secrets).each_pair { |k, v| ENV["#{k}".upcase] = v }

[matt-taylor]

Hi @girishgl

Can you please provide more context:

• What is the error message that you receive?
• What version of Ruby, Rails, Shoryuken, and Dotenv are you using?
• Are you saying that ENV["XYZ_QUEUE"] and ENV["ABC_QUEUE"] are not getting loaded in the "/app/config/shoryuken.$RAILS_ENV.yml" yml file?
• Can you also confirm that XYZ_QUEUE and ABC_QUEUE are exported ENV variables or are they only included in the .env file loaded by Dotenv?

(side note, 64 concurrency is very high)

[girishgl]

• What is the error message that you receive? 2023-03-02T05:23:26Z 17415 TID-4fvf0 WARN: No queues supplied 2023-03-02T05:23:26Z 17415 TID-4fvf0 INFO: Starting

• What version of Ruby, Rails, Shoryuken, and Dotenv are you using? ruby '2.3.1' rails '4.1.6' 'shoryuken', '5.2.1' dotenv (= 2.7.6)

• Are you saying that ENV["XYZ_QUEUE"] and ENV["ABC_QUEUE"] are not getting loaded in the "/app/config/shoryuken.$RAILS_ENV.yml" yml file? Yes

• Can you also confirm that XYZ_QUEUE and ABC_QUEUE are exported ENV variables or are they only included in the .env file loaded by Dotenv?

These are not included in .env file. these env are updated from Secret manager as below:

load environment variables from .env files require 'dotenv' Dotenv.load

require 'aws-sdk-secretsmanager' client = Aws::SecretsManager::Client.new(region: ENV['AWS_SECRET_MANAGER_REGION'] )

secrets_path = "xyz/#{ENV['RAILS_ENV']}".downcase secrets = client.get_secret_value(secret_id: secrets_path).secret_string JSON.parse(secrets).each_pair { |k, v| ENV["#{k}".upcase] = v }

• Machine configuration and queues We are currently using m6i.2xlarge EC2 machine. Total queues: 13

and we run 74 delayed job workers as well on the same machine

[girishgl]

Hi Any updates here?

[github-actions[bot]]

This issue is now marked as stale because it hasn't seen activity for a while. Add a comment or it will be closed soon.

[github-actions[bot]]

This issue was closed because it hasn't seen activity for a while.