Apply 'all' command in interactive mode twice: to scripts and to regular configs

[eugenesvk]

What exactly are you trying to do?

I have a few run_before_link config.py.tmpl scripts that maintain symlinkinkg for the folders that chezmoi doesn't manage (e.g., Application Support/AppX to .config/AppX), but after links are set up they do nothing (check that the symlink exists and exit) Now, after I edit a few regular config files in the chezmoi source folder, I'd like to then apply the changes, but carefully :), so I do chezmoi apply -v --interactive to review what changed:

What I'd like to achieve is the following:

• confirm running all the scripts on the first iteractive prompt, but only the scripts, the all flag should reset once it reaches regular configs
• individually check regular configs, e.g., using d to see diffs and skip others etc.

As far as I understand, this is currently not possible since if I reply with an a (all) command to the first script, this all 'flag' is also applied to all the regular config files. This flag, however, does seem to reset for the config X has changed since chezmoi last wrote it configs, which maintains a separate all 'flag' So basically I'd like to have the same behavior, but also for the executable scripts/other configs

What have you tried so far?

Memorize the number of scripts and press n this many times to skip all the scripts

Where else have you checked for solutions?

• [x] I have read chezmoi's user guide, and not found the answer.
• [x] I have searched chezmoi's reference guide, and not found the answer.
• [x] searched github issues

Output of any commands you've tried with --verbose flag

$ chezmoi apply -v --interactive
Apply .config/AppX/link config.py? yes/no/all/quit
# if I say 'all', then all my configs will also get overwritten

Output of chezmoi doctor

```console $ chezmoi doctor RESULT CHECK MESSAGE ok version v2.27.2, commit 882d0808feb1fc8112b411ed2216f31306656861, built at 2022-11-24T21:50:16Z, built by Homebrew ok latest-version v2.27.2 ok os-arch darwin/amd64 ok uname Darwin MacBook-Pro.local 19.6.0 Darwin Kernel Version 19.6.0: Tue Jun 21 21:18:39 PDT 2022; root:xnu-6153.141.66~1/RELEASE_X86_64 x86_64 i386 Darwin ok go-version go1.19.3 (gc) ok executable /usr/local/bin/chezmoi ok upgrade-method replace-executable ok config-file ~/.config/chezmoi/chezmoi.yaml, last modified 2022-10-16T21:26:55+07:00 warning source-dir ~/.local/share/chezmoi is a git working tree (dirty) ok suspicious-entries no suspicious entries warning working-tree ~/.local/share/chezmoi is a git working tree (dirty) ok dest-dir ~ is a directory ok umask 022 ok cd-command found /usr/local/bin/xonsh ok cd-args /usr/local/bin/xonsh info diff-command not set ok edit-command found /usr/local/bin/subl ok edit-args subl -w ok git-command found /usr/local/bin/git, version 2.38.1 ok merge-command found /Applications/4 Develop/DiffMerge.app/Contents/MacOS/DiffMerge ok shell-command found /usr/local/bin/xonsh ok shell-args /usr/local/bin/xonsh ok age-command found /usr/local/bin/age, version 1.0.0 ok gpg-command found /usr/local/bin/gpg, version 2.2.40 info pinentry-command not set info 1password-command op not found in $PATH ok bitwarden-command found /usr/local/bin/bw, version 2022.10.0 info gopass-command gopass not found in $PATH info keepassxc-command keepassxc-cli not found in $PATH info keepassxc-db not set info keeper-command keeper not found in $PATH info lastpass-command lpass not found in $PATH info pass-command pass not found in $PATH info passhole-command ph not found in $PATH info vault-command vault not found in $PATH info secret-command not set ```

Additional context

None

[twpayne]

Thank you for the well-thought through request!

As far as I understand, this is currently not possible since if I reply with an a (all) command to the first script, this all 'flag' is also applied to all the regular config files.

Yes, this is correct.

This flag, however, does seem to reset for the config X has changed since chezmoi last wrote it configs, which maintains a separate all 'flag'

This is not quite correct. chezmoi has two levels of protection here: the --interactive level is "confirm for each file" and X has changed since chezmoi last wrote it is an extra check to reduce that chance that the user's local changes to files where the changes have not been added to chezmoi's source state are not lost.

I need to think through the implications of this request. One specific difficultly is that scripts are executed at all phases: before updating your dotfiles (run_before_ scripts), while updating your dotfiles (normal run_ scripts), and after updating your dotfiles (run_after_ scripts), so introducing a way to run all run_before_ scripts, but not others feels quite specific.

Some initial suggestions:

1. Could you coalesce all of your run_before_link config.py.tmpl scripts into a single script, so you only have to skip a single script when running chezmoi apply --interactive?

2. Could you make the run_before_link config.py.tmpl scripts no-ops (i.e. empty) when the link already exists, so chezmoi doesn't even try to run them? Something like:

   {{ if not stat (joinPath .chezmoi.homeDir "Application Support" "AppX") -}}

   !/bin/sh

   ln -s ../../.config/AppX "${HOME}/Application Support/AppX" {{ end -}}

   This template evaluates to empty (i.e. chezmoi will not run it) if ~/Application Support/AppX already exists. WARNING: I have not tried this template, and the order of arguments to ln might be wrong.

3. In an extreme case, the arguments to chezmoi are available in the .chezmoi.args template variable. You could see if --interactive is included, and, if so, make your run_link_*.tmpl script templates evaluate to empty strings, so chezmoi skips them.

[twpayne]

On further thought, I would suggest using something like the following script to create your symlinks:

{{ $symlinks := dict
     "../../.config/AppX" "Library/Application Support/AppX"
-}}
{{ $symlinksToCreate := dict -}}
{{ range $target, $source := $symlinks -}}
{{   if not stat $target -}}
{{     $symlinksToCreate = set $symlinksToCreate $target $source -}}
{{   end -}}
{{ end -}}
{{ if $symlinksToCreate -}}
#!/bin/bash

{{   range $target, $source := $symlinksToCreate -}}
ln -s {{ $target }} {{ $source }}
{{   end -}}
{{ end -}}

Warning: I have not tried the above and it almost certainly contains errors. The idea is that you have a dict of symlinks that you want and you then iterate over them to find the ones that do not already exist. If, and only if, any don't exist, then you create the minimal script that creates the missing symlinks.

This gives you a single script to approve, and only in the case that you have missing symlinks.

[twpayne]

Please do report if this works for you. If this approach works, then it's a worthwhile FAQ entry as is it's an intermediate solution until #2273 is implemented.

[eugenesvk]

so introducing a way to run all runbefore scripts, but not others feels quite specific

I think run_after would also be fine (though I don't use them, so maybe misunderstand their common purpose), and this specificity I think is justified by the difference in their behaviour: run_ scripts are commonly used to update your dotfiles (e.g., I use them to ignore a line with a font size, but otherwise track all other config changes), while run_before/_after are commonly used for some other purposes So treating regular run_ scripts like regular config files (and don't have a separate all flag) seems fine?

Could you coalesce all of your run_before_link config.py.tmpl scripts into a single script, so you only have to skip a single script when running chezmoi apply --interactive?

Then I'd lose the 'locality' of those configs. Currently they're all nicely grouped in a single folder (and after I've learned about includeTemplate "./private_dot_config/AppX/appx_def.tmpl" I put the app templates in the app config folder as well to avoid the need to remember in a few months that for this app the actuall configs are in a special template folder) (although on second thought I might be able to restructure the scripts from stand-alone executables to have only app-specific function and then include them in your suggested single script, will think about that)

Could you make the run_before_link config.py.tmpl scripts no-ops (i.e. empty) when the link already exists, so chezmoi doesn't even try to run them? Something like:

This doesn't seem to work (even after adding a few more - to avoid any spaces), with the script below I still get prompted whether I'd need to run this script, which then does nothing when I say y(yes), so I know it works, the condition is true, no 111 is printed

{{- if not (stat (joinPath .chezmoi.homeDir)) -}}
#!/bin/sh

echo 111
{{- end -}}

But if it worked, this would be a good solution, adding cross-platform logic shouldn't complicate this much

In an extreme case, the arguments to chezmoi are available in the .chezmoi.args template variable. You could see if --interactive is included, and, if so, make your runlink*.tmpl script templates evaluate to empty strings, so chezmoi skips them.

But I do need those checks (what if the symlink was overwritten by the app (like mentioned in this https://github.com/twpayne/chezmoi/issues/1348#issuecomment-900503924 ), I'd like to interactively learn about it)

On further thought, I would suggest using something like the following script to create your symlinks:

I have the same issue as with the script above — I still get a prompt to run the script even if the stat call is successful and the script is supposed to be blank

[eugenesvk]

If this approach works

I like your "empty template" approach (for each config file) best, that's the least 'disruptive' to my current configs, so it would solve the issue in an even better way than what I've suggested as it'd allow more granularity

[twpayne]

Did the empty template approach work for you?

[eugenesvk]

Unfortunately not, chezmoi still asks about these empty templates (and then does nothing as expected), I just wanted to add the if it worked, it would be the best option

[twpayne]

Could you post the exact contents of your template please?

[eugenesvk]

But I already have in the comment above

This doesn't seem to work (even after adding a few more - to avoid any spaces), with the script below I still get prompted whether I'd need to run this script, which then does nothing when I say y(yes), so I know it works, the condition is true, no 111 is printed

{{- if not (stat (joinPath .chezmoi.homeDir)) -}}
#!/bin/sh

echo 111
{{- end -}}

[twpayne]

This doesn't seem to work (even after adding a few more - to avoid any spaces), with the script below I still get prompted whether I'd need to run this script, which then does nothing when I say y(yes), so I know it works, the condition is true, no 111 is printed

{{- if not (stat (joinPath .chezmoi.homeDir)) -}}
#!/bin/sh

echo 111
{{- end -}}

Does the file containing this have a .tmpl extension?

[eugenesvk]

yes, for example, run_before_atest.sh.tmpl and I still get Apply .config/bash/atest.sh? yes/no/all/quit on chezmoi apply -v --interactive

[twpayne]

Ah, now I see. This only occurs if --interactive is specified. #2612 fixes it.

[eugenesvk]

Thanks for a quick fix, can confirm it's working now and with just one extra template condition it's also easy to add

Just a tip for future readers :): if you need to check that both folders should exist (=either of the two folders missing), you can do the following with an extra template variable dstdir:

{{ $dstdir := print (joinPath .chezmoi.homeDir "Library/Application Support/AppX/Packages") -}}
{{ if or (not (stat (joinPath $dstdir "Default"))) (not (stat (joinPath $dstdir "User"))) -}}

#!/bin/sh

echo "Missing Either ~/Library/Application Support/AppX/Packages/Default or ~/Library/Application Support/AppX/Packages/User"

{{ end -}}

and to use environment variables (env comes from sprig):

{{ $dstdir := (env "CARGO_HOME") -}}

[eugenesvk]

Sorry, but after some more testing I realize there is a mistake in the template logic — it skips not only symlinks as targets, but also regular folders. But if the target is a regular folder, I need to print a warning since the app setup is wrong, e.g., the target configs haven't been moved to the chezmoi source folder, deleted, and symlinked (or maybe the app has overwritten the symlink or something else happened) So I need the template to only check if the target exists AND is a symlink

As far as I understood, stat can't check if the path is a symlink since it follows symlinks, but then which template function can differentiate between symlinks and regular paths?

[twpayne]

As far as I understood, stat can't check if the path is a symlink since it follows symlinks, but then which template function can differentiate between symlinks and regular paths?

At the moment, there is no such function, although I will add an lstat function template function that does this.

In the meantime, you can use {{ $fileType := output "stat" "--format" "%F" $filename | trim }}.

[eugenesvk]

Thanks, this works! Another tip: to avoid having the template symlink check throw errors at you when the folder doesn't exist (as then stat would error) add the 'folder exists' condition before the 'folder is a symlink' condition (then if no folder exists, the symlink condition will not be evaluated)

{{ $dstdir  := joinPath .chezmoi.homeDir "Library/Application Support/AppX/Packages" -}}
{{ $dstcfg1 := joinPath $dstdir "Default" -}}
{{ $dstcfg2 := joinPath $dstdir "User" -}}
{{ if
or (or (not (stat $dstcfg1))
       (ne  (output "stat" "--format" "%F" $dstcfg1 | trim) "symbolic link")  )
   (or (not (stat $dstcfg2))
       (ne  (output "stat" "--format" "%F" $dstcfg2 | trim) "symbolic link")  )
-}}

[twpayne]

2599 adds an lstat template function.

[eugenesvk]

Thanks a bunch! Couldn't check the brew head version due to a lack of v1.2.3 versioning(?) there, get chezmoi: HEAD is not in dotted-tri format, so will just wait for a release

[twpayne]

You can grab the latest binary from https://github.com/twpayne/chezmoi/actions/runs/3624270200#artifacts

[eugenesvk]

Thanks, this one works and doesn't generate the error that homebrew-ed built binary generates Can confirm that this works and better than the previous one as you don't need to check that the folder exists anymore, the template function fails gracefully