Environment variables file for "balena push <ip-address>"

[zrzka]

--env argument is nice and works. Ability to specify path to the env file from where it loads & sets all environment variables instead of this would be superb.

[zrzka]

I know that I can set them in .resin-sync.yml, but it's not convenient because this file is owned by the root and also how my script works - I rsync source to the temporary folder, because local push --ignore doesn't work for me, etc. It's just speed up optimisation.

[imrehg]

Ping to trigger jellyfish import

[balena-ci]

[imrehg] This issue has attached support thread https://jel.ly.fish/#/support-thread~d3009c73-8802-425a-a14f-b284334d3847

[balena-ci]

[imrehg] This issue has attached support thread https://jel.ly.fish/#/support-thread~036414ce-afcd-4a99-a49d-b252ac5f3b8c

[danielmahon]

Copied from my comment to this forum post (in-case it helps anyone or sparks an idea for integration): https://forums.balena.io/t/local-mode-environment-variables/17500/23

I am currently handling this on Mac OS by using an inline npm script to grep a .env file and pass that through xargs to the balena push command. Works well at the moment and closely mimics my local dev setup. Using a multi-container setup with app and server services I just put a simple package.json file next to my docker-compose.yml file with:

project folder

.
├── .env
├── .git
├── .gitignore
├── app
├── docker-compose.yml
├── package.json
└── server

package.json

...
  "scripts": {
    "local": "grep -v '^#' .env | sed 's/^/--env /' | xargs balena push development.local",
    "deploy": "balena push my-application"
  },
...

.env

# Service specific
app:APP_HEIGHT="1920"
app:APP_WIDTH="1080"
# Device wide
DISPLAY_PRIMARY_ID="DP-1"
DISPLAY_SECONDARY_ID="DP-2"
DISPLAY_PRIMARY_ROTATION="right"
DISPLAY_SECONDARY_ROTATION="normal"

Now yarn local or npm run local will read all uncommented lines from .env, append --env to each line, and pass the arguments through to balena push.

By keeping this file in the root folder you can still keep local .env files in their respective service folders (app, server) for local development if that's how you choose to load ENVs in development.

[pdcastro]

Related to #441, #656, #1255

[balena-ci]

[karaxuna] This issue has attached support thread https://jel.ly.fish/#/support-thread~8780fa8c-06d7-4949-b978-0aa56d0a52ef

[balena-ci]

[ffissore] This issue has attached support thread https://jel.ly.fish/#/144dcd93-c853-4cfb-9026-099c651dad02

[ghost]

What's the latest on this? I can see a few issues mentioning it and some posts on the forum but seems to be no activity since late 2019.

Being able to specify a .env in the same directory as docker-compose.yml and have that picked up on for local push seems to be the most suitable solution that many are waiting for, that would avoid the need to manually edit Dockerfiles and docker-compose.yml before deploying to production or write long dev commands with lots of --env flags for local development.

Additionally, for people like me who have a specific development application and device, the ability to generate such a .env from existing variables defined for the device in the dashboard (and also potentially do the reverse i.e. run a cli command that will automatically generate the variables in the dashboard from a local .env) would be really useful.

This would avoid issues such as accidentally leaving env variables in files committed to source control, forgetting to update a local push command, having to specify a long local push command (that might be written to an npm script for convenience and again might accidentally end up in a source repository).

As an example of a similar situation, Firebase has a nice way of dealing with this for Cloud Functions - env variables are created in the cloud environment via the cli with a command such as firebase functions:config:set VAR1=value1 VAR2=value2, then when running functions locally, simply firebase functions:config:get > .runtimeconfig.json, then .runtimeconfig.json gets used for serving functions locally and can simply be included in .gitignore so there's no risk of committing API keys etc. to source control. There's no need to manually edit other files or commands or have special logic in source code to check for the running environment and use different env files etc. As a bonus, they also have a method for cloning env/configuration variables between projects (something like this would be great for being able to clone env variables between applications on Balena Cloud programmatically).

[pdcastro]

@timelapsepaul, the suggestion of reading env vars from the web dashboard (in addition to an env var file, or in order to pre-populate it) is a good one, thanks! 👍

I am bumping the priority of this issue in the backlog. Meanwhile, in case this hint is helpful, you could create a local-var.env file with contents such as:

--env var1=value1
--env "var2=value 2"
--env var3=v3

And then, using the bash shell, you could invoke the CLI with:

$ cat local-vars.env | xargs balena push <ip-address>

To visualize the command produced by xargs, add 'echo' to the command line:

$ cat local-vars.env | xargs echo balena push 192.168.0.37
balena push 192.168.0.37 --env var1=value1 --env "var2=value 2" --env var3=v3

On Windows, a bash shell is available with options such as MSYS, MSYS2, Git for Windows and the Microsoft Windows Subsystem for Linux, which are further described in the CLI documentation section "Choosing a shell".

(This hint is actually a variation / extract of the comment by @danielmahon above.)

[ghost]

On another note (I don't know if there's another issue for this) I did notice the latest version of the cli includes a note about using the --no-gitignore (-G) flag to ignore .gitignore and only use .dockerignore. But still with this flag, my .env file does not get copied to the container.

I've tried the .env and .dockerignore (which does not list .env) file in both the root with my docker-compose.yml, and my service source folder with no luck. Am I missing something or does this not actually work yet, or not supported for local development?

[pdcastro]

I've tried the .env and .dockerignore (which does not list .env) file in both the root with my docker-compose.yml, and my service source folder with no luck. Am I missing something or does this not actually work yet, or not supported for local development?

This GitHub issue / feature request (Environment variables file for "balena push ") has not been implemented yet (this issue remains open). My previous comment, which mentioned a local-var.env file, was a suggestion of using the xargs utility on a bash shell as a workaround while this issue is not implemented.

[SpencerWF]

Has there been any further progress on this? I have been using the solution as described by @pdcastro, but I have just turned by device from a single-container device into a multi-container device and seem to have an issue crop up. When I first push to the local device the environment variables come fine from the cat command. However, if I do a change to a service's Dockerfile.template and the service rebuild gets pushed then the environment variables don't remain. This does not appear to happen if the change occurs to other files in the service.

[pdcastro]

if I do a change to a service's Dockerfile.template and the service rebuild gets pushed then the environment variables don't remain

@SpencerWF, this sounds like an instance of issue #1965 Livepush: Environment variables passed via the CLI are lost after a change to a Dockerfile

[bbugh]

Another possible option would be using the already existing .balena/balena.yml functionality, adding a new separate section for "local" mode environment variables. The CLI requires a lot of different switches in different modes, which makes it a lot harder for devs that are less familiar with balena to use it. This would also help guarantee that problematic switches like --multi-dockerignore are correctly passed every time.

[frederikheld]

Hey @pdcastro, any progress on this? I just ran into the issue that fleet vars are not being applied in local mode and came up with the idea of using an .env file but it didn't work. After some research I found this issue and was happy to see that others are suggesting this as well. I would really appreciate to see this feature soon! Parsing an .env file would be in line with the docker-compose approach, so I would prefer it over other approaches mentioned here.

[jrc]

+1

That said, here's a way to do it with a "normal" .env file, i.e. without the explicit --env on each line:

cat .env | xargs -I@ echo "--env @" | xargs balena push ${DEVICE_IP}

[thirstyfish]

+1

That said, here's a way to do it with a "normal" .env file, i.e. without the explicit --env on each line:

cat .env | xargs -I@ echo "--env @" | xargs balena push ${DEVICE_IP}

To filter out comment lines in the .env file starting with #:

cat .env | grep "^[^#;]" | xargs -I@ echo "--env @" | xargs balena push ${DEVICE_IP}