Set global flags using environment variables

[lcrockett]

Is your feature request related to a problem? Please describe.

When working with different chezmoi source directories on the same machine it is useful to be able to run a chezmoi command that is influenced by environment variables. For instance, direnv can be used to set different "cache | config | source" directory environment variables that chezmoi will use when executed.

This allows one to cd to alternative source directories and being allowed to use chezmoi diff, chezmoi apply, etc. while in the alternative directory and actually use that same source directory as a source path, and other alternative directories for the cache and configuration. This will prevent the necessity to always use --cache | --config | --source with every invocation of chezmoi when using alternative directories.

Describe the solution you'd like

Have chezmoi obey environment variables to influence the used cache | config | source directories. For instance CHEZMOI_CACHE_PATH | CHEZMOI_CONFIG_PATH | CHEZMOI_SOURCE_PATH.

Describe alternatives you've considered

Alternatives are to add a local .bin directory to $PATH with a script called chezmoi that executes chezmoi with the alternative directories set as arguments and then have direnv add that .bin directory automatically upon entering a chezmoi source directory. Works, but using environment variables are more suited for this scenario and easier to maintain.

[halostatue]

What at interesting feature request. It shouldn’t be hard to implement, although it would be necessary to make sure that the command-line parameters override the environment variables if both are specified.

Can you describe how you would use multiple chezmoi repositories like this? I’d like to understand this use case a bit better to see if there’s a different feature that might be worth considering for a future release.

[lcrockett]

Scenario

4 different workstations, 3 different "users / usernames / home directories" (they're all me, private | work | work client).

Issue

Too much duplicated data (neomutt, ZSH, Neovim, GnuPG, TMUX, etc. etc.)

Solution

• ~/.local/share/chezmoi == private dotfiles linked to private git repository per user (so 3 separate dotfiles repositories in total, as is the default mode of using chezmoi)
• ~/.local/share/chezmoi-global == global dotfiles git repository containing all duplicated data (so 1 repository for all 3 users, all 3 users can commit / pull from this single repository)

Execution (order)

1. I run chezmoi --cache "$HOME/.cache/chezmoi-global" --config "$HOME/.config/chezmoi-global/chezmoi.yaml" --source "$HOME/.local/share/chezmoi-global" <whatever> 1a. This gives me all globally shared configuration
2. I run chezmoi --cache "$HOME/.cache/chezmoi" --config "$HOME/.config/chezmoi/chezmoi.yaml" --source "$HOME/.local/share/chezmoi" <whatever> OR chezmoi <whatever>. I'd personally prefer the former in order to be explicit. 2a. This gives me all personal scoped configuration

Practice

• I execute chezmoi (update / apply) in order with the help of a custom ZSH function. This is to ensure global is always applied first, and personal is applied second. This also ensures any accidental duplicates in global + personal repositories will be overridden by the personal repository
• I have to make sure myself I don't manage the same file in both repositories. So far I didn't die and it works flawlessly. Also, why would I want to do that

What would this fix ?

• When I change chezmoi repository contents, wether that's the global or personal repository / directory, i'm always executing a quick chezmoi diff, or chezmoi apply on one system where i'm changing / developing configuration changes until i'm satisfied and ready to commit. I'd like to be able to do this without using temporary aliases or writing out the full chezmoi command with all necessary arguments. Instead, i'd have direnv take care of setting the appropriate environment variables that chezmoi would then pick up on
• Think of changing configuration for something like Neovim and wanting to try out changes locally before committing. I used to have 3 synchronised TMUX panes open and making changes in the context of the 3 different users at the same time. No thanks

Having command-line parameters override environment variables would certainly be the standard way tools deal with order. Good catch

[halostatue]

I’ve added a draft PR that should work. There’s no tests, barely any documentation, and I haven’t even run it myself to see if it Does The Right Thing. I’ll get to that later, but I think that this should work.

I’m sure you know this, but because I ran into this multiple times‡ myself recently (and I consider myself a direnv expert…), remember that direnv typically only updates on prompt display, so you would need to run direnv exec DIR COMMAND [...ARGS] in your wrapper script to ensure that direnv updates are applied.

In my case I was doing for d in env-*; do cd $d && terraform plan -out plan && cd ..; done but needed to do for d in env-*; do cd $d && direnv exec . terraform plan -out plan && cd ..; done. Oops.

[twpayne]

Thanks for the detailed info @lcrockett. A few questions/comments:

1. chezmoi is designed to generate dotfiles for multiple machines/configurations from a single source of truth. Is there a reason that you can't merge the dotfiles from the three "all you" users into a single repo?

2. The cache and source directories can be set in chezmoi's configuration file. With this, you only need to specify -C to chezmoi to switch all directories.

3. Currently chezmoi sets various CHEZMOI_ environment variables itself (#2429, #2743). I'm concerned that there will be some subtle and possibly bad interactions between these and chezmoi also reading environment variables. At least some careful thought is required so that running chezmoi cd and then another chezmoi command has the desired effect.

[halostatue]

@twpayne, I’m not @lcrockett, but I can think of places where I might use this:

1. I currently mix work and personal configuration and would consider using something like this to split the configuration.
2. I think that having $CHEZMOI_CONFIG would still be useful for integration with direnv and not having to remember to supply -C. I’m going to be doing some work with #2773 and will probably push the documentation in that direction (I’ll try to update the documentation as its own commit so that it can be separated off later).
3. The changes in #2773 do not overlap those environment variables, but only with the descriptions in assets/chezmoi.io/docs/user-guide/advanced/use-chezmoi-with-watchman.md.

https://github.com/halostatue/chezmoi/blob/d83b5e9644b2774b220773ef63663fb5800d5f98/pkg/cmd/config.go#L1478-L1483

[lcrockett]

@twpayne

1. The global repository already generates dotfiles for multiple scenario's based on user and workstation metadata. There is however secretive metadata (that are not secrets) involved in workstation configuration of work clients that I can't just push to any repository, this data has to stay on-premise and so i'm bound to separating that specific data in separate dotfiles repositories. Additionally I can't necessarily always access these git endpoints of work clients due to reasons, having an external globally accessible git endpoint with the global dotfiles repository helps in that sense so i'm always able to change once and commit once for all scenario's / users
2. That's a good point, I didn't think of checking if a configuration file would help. That would make it already easier indeed
3. Just to clarify, i'm not proposing chezmoi picks up on additional variables when running chezmoi cd (never really used this option), i'm proposing chezmoi to pick up on additional variables when it's run in a regular shell. In my use case I would always already sit in the appropriate directory (~/.local/share/chezmoi or ~/.local/share/chezmoi-global)

@halostatue Thank you for the PR, I could test it out once you're done with it. The $CHEZMOI_CONFIG proposal sounds good. Also, i'm aware of the direnv specifics you've mentioned. In this specific case I would always sit in an interactive shell and any directory change would trigger a prompt display anyways. Cheers

[Lockszmith-GH]

Interesting, I implemented a system of my own to solve what you are proposing. I have a bunch of scripts:

• cz - aka default
• czb - aka base
• czx - aka externals

each of those work in a different context - cz and czb are working with a different config path and in different git repos. czx is just a different configuration within the czb repo.

cz initializes czb via run_ and run_once_ scripts, and czb initializes czx.

My different chezmoi based scripts use an environment variable named CONTEXT. CONTEXT when missing, defaults to cz, but it can be set to czb or czx to interact with that different context.

I too use direnv to manipulate the default context, depending on my work, and so switching CONTEXT does that.

My scripts look like this: cz

alias cz="chezmoi"

czb

[[ -z "$CZ_B" ]] && echo "CZ_B isn't defined!" && exit 1

chezmoi --source "$CZ_B" --config ${USER_HOME:-$HOME}/.config/chezmoi/_base/chezmoi.toml "${@}"

czx

[[ -z "$CZ_X" ]] && echo "CZ_X isn't defined!" && exit 1

chezmoi --source "$CZ_X" --config ${USER_HOME:-$HOME}/.config/chezmoi/_externals/config.toml "${@}"

paired with the following aliases:

alias Cb="CONTEXT=czb "
alias Cx="CONTEXT=czx "

As you can see the variation is in both --source location and --conifg location.

A good example where the concept of CONTEXT comes into play is my cz-get-data script. cz-get-data:

#! /usr/bin/env bash
CONTEXT=${CONTEXT:-cz}
QUERY="${@:-my_data_root_name_here}"
[[ $QUERY == "." ]] && QUERY=""
${CONTEXT} data --format json | jq -r ".${QUERY}"

Or cz-init script (which calls the above cz-get-data):

#! /usr/bin/env bash
export CONTEXT=${CONTEXT:-cz}
$CONTEXT init --config-path "$(cz-get-data chezmoi.configFile)" ${@}

I wrote cz-init because just using a different CONTEXT for the init wasn't enough, as the --config-path argument was required.

I invoke it from command line like this:

CONTEXT=czb cz-init
# or
Cb cz-init

The layered approach you are describing, will allow me to do this without all of the scripting infrastructure.

[twpayne]

I just wrote in https://github.com/twpayne/chezmoi/pull/2773#issuecomment-1462700844:

Sorry, I am against this [PR].

Currently chezmoi sets a wide range of environment variables for processes that it spawns. All of these variables begin with CHEZMOI.

This PR adds a special environment variable which is the first and only one to be read by chezmoi. This is inconsistent with chezmoi's existing use of environment variables.

I don't like configuration being set via environment variables as they are generally not trivially visible to the user (unlike, say command line arguments). I think the original problem is better solved with a wrapper script or alias.