Add system-wide env file

[ericfranz]

Perhaps /etc/ood/config/env would be a file containing an environment variable. Then we could do Dotenv.load(Pathname.new("/etc/ood/config/env")) in an initializer.

Would take some thought. Ideally, this would be the first file, followed by the others. In that case, it would make more sense to avoid doing require now and have this be in an initializer. We could load everything in an initializer

https://github.com/bkeepers/dotenv/blob/master/lib/dotenv/rails.rb: Dotenv::Railtie.load is here:

def load
  Dotenv.load(
    root.join(".env.local"),
    root.join(".env.#{Rails.env}"),
    root.join(".env")
  )
end

This will get called during the before_configuration callback

So we could actually do our own version i.e.

Dotenv.load(
  Pathname.new("/etc/ood/config/env"),
  root.join(".env.local"),
  root.join(".env.#{Rails.env}"),
  root.join(".env")
)

Note there is also a Dotenv.overload that overwrites env vars already set. Then rake tasks would pick this up too.

This would be a place for custom shared branding (think BOOTSTRAP overrides) and OOD_PORTAL and OOD_DASHBOARD_TITLE. We might consider this an option for even setting a default OOD_DATAROOT.

This could be a solution for https://github.com/OSC/ood_appkit/issues/30 as well.

[nickjer]

I think a yaml file may be more appropriate, i.e., /etc/ood/config/portal.yml.

But this will require a design meeting.

[ericfranz]

Using a portal.yml file, you can add defaults that include hashes and arrays, which could be overridden by configuration. Similar to clusters.

[ericfranz]

With a .env we can have bash code run however.

[ericfranz]

One of the benefits would be easier configuration and installation of apps in OOD.

[nickjer]

Does the node.js dotenv interpolate Bash variables in the .env file? If not, then using a global .env won't be as useful as a plain old yaml file.

[ericfranz]

Another option is if we use an env file, the wrapper scripts could source this file prior to executing ruby or python etc. So these env vars would be set prior to any use of dotenv.

[brianmcmichael]

In the case of node, it looks like there is an addon that gets us variable expansion like we use in dotenv-rails ~>2.1. https://github.com/motdotla/dotenv-expand

[nickjer]

@ericfranz: you can't give your own comment a thumbs up, but... I do like your idea

[nickjer]

Only issue would be that the env vars wouldn't be set if running rails console, but that may not be too big of an issue.

[ericfranz]

Solution proposal:

1. The env file would be located by default at /etc/ood/config/env
2. For PUNs the env is sourced in the wrapper scripts or better yet sourced in the script that inits the PUN
3. For Rails console and rake tasks, we can add the sourcing this env file as part of the requirement for properly running these commands => at least for the precompile assets step. When building assets through the dashboard, these env vars will already be part of the environment, even when running Bundler.with_clean_env

[brianmcmichael]

Only issue would be that the env vars wouldn't be set if running rails console, but that may not be too big of an issue.

This would be a big issue. Any envvars would also need to be loaded in during the rake test process.

[nickjer]

@ericfranz: Before we decide on where to hook into the PUN loading chain, maybe we should come up with a draft env file that we intend to source. This will better describe how early or late we want to source this file.

[ericfranz]

The most immediate concern are setting environment variables to remove the need for custom .env files per site i.e. OSC OnDemand and AweSIM. This will address issues like https://github.com/OSC/ood-myjobs/issues/169. One key is that for these env vars rake tasks will need to be able to source the appropriate env file and we need to be careful when running bin/setup-production or other tasks from another OOD app (like Developer views, or an OOD shell app).

While I'd quickly suggest OOD_PORTAL (ondemand, awesim) and OOD_SITE (osc, awesim) we might want to be aware of a possible future where we get rid of two separate "apps.awesim.org" and "ondemand.osc.edu" and just have one portal with the same app list etc.

That said, here is a short list of env vars that we commonly use in apps and that require a .env.local.

# portal specific
OOD_DASHBOARD_TITLE
OOD_DASHBOARD_URL # (ever used?)
OOD_PORTAL #i.e. ondemand, awesim ==> affects where data and dev apps are installed
OOD_SITE #i.e. awesim, osc ==> for suffixes for site specific .env and initializer files

Below are other options

App specific

APP_TOKEN
RAILS_RELATIVE_URL_ROOT # could omit if we start versioning assets
OOD_DATAROOT

Other ood_appkit urls (but should we still keep using this? seems to conflict with the idea of finding an app deployed who plays the role of shell or files)

OOD_SHELL_URL
OOD_SHELL_TITLE  # never used
OOD_EDITOR_URL
OOD_EDITOR_TITLE  # never used
OOD_FILES_URL
OOD_FILES_TITLE # never used

For the dashboard (or an app index):

# path to nginx_config yaml file for determining where apps are installed, etc.
OOD_NGINX_CONFIG

[ericfranz]

We could just start with these first...

# portal specific
OOD_DASHBOARD_TITLE
OOD_DASHBOARD_URL # (ever used?)
OOD_PORTAL #i.e. ondemand, awesim ==> affects where data and dev apps are installed
OOD_SITE #i.e. awesim, osc ==> for suffixes for site specific .env and initializer files

# path to nginx_config yaml file for determining where apps are installed, etc.
OOD_NGINX_CONFIG

[nickjer]

I am confused about the difference between OOD_PORTAL and OOD_SITE. Is there is a case where they could be different?

Also, OOD_NGINX_CONFIG may not be helpful as in most cases the yaml file will be all commented out or empty.

Where do we store this source file? For example:

/etc/ood/env

Where is this injected into the code? I suggest ood_appkit requires dotenv and directly sources this:

global_env = Pathname.new( ENV['OOD_GLOBAL_ENV'] || '/etc/ood/env' )
Dotenv.load(global_env) if global_env.file?

that way it will work for rake and rails calls from the command line out of the box. Also, Bundler.with_clean_env will clean these env vars up for fork'ed processes.

[ericfranz]

Using https://github.com/OSC/ood-myjobs/tree/36608f90957504d7f19f08ff332de7d8f433554f as an example.

• OOD_PORTAL may be either "ondemand" or "awesim" and is used to build the dataroot in .env.production files i.e. OOD_DATAROOT=$HOME/$OOD_PORTAL/data/$APP_TOKEN
• OOD_SITE may be either "osc" or "awesim" and is used for OSC specific configuration by suffixing the name of .env.local files and initializers so during the app build we copy those files to configure the apps for OSC/AweSim
  • we could change the way we use OOD_SITE so that whether for AweSim or OnDemand installs OOD_SITE would only be "osc"; then if we were using files or custom code we might create a suffix like $OOD_SITE.$OOD_PORTAL i.e. .env.local.osc.ondemand and .env.local.osc.awesim

I'm fine with /etc/ood/env or /etc/ood/config/env

When to inject it into the code? Not sure yet.

[nickjer]

OOD_SITE may be either "osc" or "awesim" and is used for OSC specific configuration by suffixing the name of .env.local files and initializers

So why couldn't it also be used to suffix .env.production files and have...

# .env.production.osc

OOD_DASHBOARD_TITLE="OSC OnDemand"
APP_TOKEN=sys/myjobs
OOD_DATAROOT=$HOME/ondemand/data/$APP_TOKEN
DATABASE_PATH=$OOD_DATAROOT/production.sqlite3
RAILS_RELATIVE_URL_ROOT=/pun/$APP_TOKEN

# .env.production.awesim

OOD_DASHBOARD_TITLE="AweSim Apps"
APP_TOKEN=sys/myjobs
OOD_DATAROOT=$HOME/awesim/data/$APP_TOKEN
DATABASE_PATH=$OOD_DATAROOT/production.sqlite3
RAILS_RELATIVE_URL_ROOT=/pun/$APP_TOKEN

as OOD_PORTAL seems to really only be a local variable used to fill in OOD_DATAROOT.

[nickjer]

I feel the .env.local.osc shouldn't be used. As development should be done under the default OOD install environment, and if you want to test new features you can locally add a .env.local with modifications but don't commit it.

[nickjer]

Maybe I misunderstood initially. Is this more of what you want?

# .env.production

APP_TOKEN=sys/myjobs
OOD_DATAROOT=$HOME/$OOD_PORTAL/data/$APP_TOKEN
DATABASE_PATH=$OOD_DATAROOT/production.sqlite3
RAILS_RELATIVE_URL_ROOT=/pun/$APP_TOKEN

# .env.local.osc

OOD_DASHBOARD_TITLE="OSC OnDemand"

# .env.local.awesim

OOD_DASHBOARD_TITLE="AweSim Apps"

where OOD_PORTAL is set before the call here:

# Gemfile

...
gem 'dotenv-rails', '~> 2.0', require: 'dotenv/rails-now'
...

[ericfranz]

Determined offline that any shared env we set in a file like /etc/ood/env or /etc/ood/config/env needs to be true across all apps and all environments (development mode, production mode, etc.).

[ericfranz]

Two options discussed:

A. ood_appkit loads the /etc/ood/env

-gem 'dotenv-rails', '~> 2.0', require: 'dotenv/rails-now'
+gem 'ood_appkit', '~> 0.3.2', require: 'ood_appkit/rails-now'

And add a file ood_appkit/lib/rails-now where we do a little more than what dotenv/rails-now does. This would load the env from /etc/ood/env and then call dotenv/rails-now.

Benefits:

1. Rake tasks and running Passenger instances both work (i.e. precompiling assets would auto-load env vars appropriately)

Drawbacks:

1. Would need a separate implementation to pull vars in for node and python apps
2. Would need separate implementation for ruby apps that do not use ood_appkit gem

B. Passenger ruby/node/python wrapper scripts source the /etc/ood/env file

We won't do this in the ood_ruby script that is used to execute nginx_stage because that is executed as root. However, we could put it in /etc/ood/env or the Passenger ruby + python + node wrapper scripts load the /etc/ood/env

Benefits:

1. env vars are available to all running Passenger instances of apps immediately

Drawbacks:

1. without explicitly sourcing the env file on the command line, env vars are not available to apps when running rake tasks or rails console (precompile assets)
2. when interacting with apps through dashboard or another developer tool (executing shell commands on localhost, etc.), env vars set in the app you are working through would not be cleared by Bundle.with_clean_env because they would be set before hand so if we needed these env vars to be different we would need to unset them
3. knowledge of env file exists in multiple places

Either way, we want to avoid user specific env vars in this file when running rake tasks. Or if user specific env vars were set, we shouldn't compile those into the builds. i.e. $HOME or $USER.

[ericfranz]

This is out of date. We now can set env's for all of OnDemand using nginx_stage. We are going to have centralized config that all the apps use as well in the form of YAML.