[Feature request] Variables in separate yaml

[mchwalisz]

I wold like to split variables (and/or dynvariables) to a separate file. Yes I know that I could use environment variables to move my way around it, but that for me looks like workaround not a clean solution.

My idea was to provide in config section a variables key, which (if present) would point to filename from where the rest of variables can be taken. If the key is not there (or set to None) we go back to standard behavior. Actually, it should be pretty easy to merge variables from config.yaml and from new file (making one of them priority in case of duplicates).

To give some perspective. I'm using git crypt to encrypt sensitive data (like variables) and put them into git repository. This allows for fast recovery of dotfiles. I clone my dotfiles repo, I put gpg key on the machine and I can unlock everything. Unfortunately, currently I need to encrypt either env variables (which makes usage of dotdrop more cumbersome) or encrypt config.yaml (my current approach, but I cannot have partial solution where I deploy non encrypted stuff as main config is locked).

I understand it can be low priority.

[deadc0de6]

So if I'm getting this right, you'd like to do something like that:

config.yaml

variables:
  v1: var2
config:
  include_file_variables: 
  - ext_variables.yaml

ext_variables.yaml

variables:
  v1: var1
dynvariables:
  dv1: "echo test"

Then depending on which sources gets the priority, v1 would have value var1 or var2.

[mchwalisz]

Exactly.

[deadc0de6]

That's doable, I'll look into it and see what I can do.

[deadc0de6]

I have made an attempt at external variables in the branch extvariables.

This should allow you to have variables retrieved from an external file:

config.yaml

config:
  backup: true
  create: true
  dotpath: dotfiles
  include_variables:
  - variables.yaml
variables:
  var1: "var1"
  var2: "{{@@ var1 @@}} var2"
  var3: "{{@@ var2 @@}} var3"
  var4: "{{@@ dvar4 @@}}"
  varx: "test"
dynvariables:
  dvar1: "echo dvar1"
  dvar2: "{{@@ dvar1 @@}} dvar2"
  dvar3: "{{@@ dvar2 @@}} dvar3"
  dvar4: "echo {{@@ var3 @@}}"
dotfiles:
  f_abc:
    dst: ~/.abc
    src: abc
profiles:
  p1:
    dotfiles:
    - f_abc
    variables:
      varx: profvarx

variables.yaml

variables:
  var1: "extvar1"
  varx: "exttest"
dynvariables:
  dvar1: "echo extdvar1"

The paths from include_variables is either relative to the config.yaml or absolute.

Here is the order of priority:

profile variables > external variables > local variables

But of course you could have all your variables defined in the external file only.

The above results in following variables defined:

• 'var2': 'extvar1 var2'
• 'var4': 'echo extvar1 var2 var3'
• 'dvar3': 'extdvar1 dvar2 dvar3'
• 'dvar4': 'extvar1 var2 var3'
• 'dvar1': 'extdvar1'
• 'varx': 'profvarx'
• 'var3': 'extvar1 var2 var3'
• 'profile': 'p1'
• 'dvar2': 'extdvar1 dvar2'
• 'var1': 'extvar1'

Could you give it a try and let me know if that solves your use-case? Thanks!

[mchwalisz]

For now it works as expected. It nicely fails on encrypted dotdropvars.yaml and provides empty variables (as without import the don't exist).

$ git crypt lock 
$ dotdrop install
[ERR] 'utf-8' codec can't decode byte 0xd8 in position 10: invalid continuation byte 
[WARN] "/home/vene/.dotfiles/dotfiles/dotdropvars.yaml" does not exist 
[ERR] source dotfile does not exist: /home/vene/.dotfiles/dotfiles/config/autorandr_ 
[ERR] source dotfile does not exist: /home/vene/.dotfiles/dotfiles/./config/sublime-text-3/Packages/User/de_DE.dic 
diff "/home/vene/.dotfiles/dotfiles/./config/sublime-text-3/Packages/User/Slack.sublime-settings" VS "/home/vene/.config/sublime-text-3/Packages/User/Slack.sublime-settings"
3c3
<       "TKN": ""
---
Overwrite "/home/vene/.config/sublime-text-3/Packages/User/Slack.sublime-settings" [y/N] ? 
[WARN] ignoring /home/vene/.config/sublime-text-3/Packages/User/Slack.sublime-settings 
diff "/home/vene/.dotfiles/dotfiles/./config/sublime-text-3/Packages/User/MarkdownPreview.sublime-settings" VS "/home/vene/.config/sublime-text-3/Packages/User/MarkdownPreview.sublime-settings"
2c2
<     "github_oauth_token": "",
---
Overwrite "/home/vene/.config/sublime-text-3/Packages/User/MarkdownPreview.sublime-settings" [y/N] ? 
[WARN] ignoring /home/vene/.config/sublime-text-3/Packages/User/MarkdownPreview.sublime-settings 
diff "/home/vene/.dotfiles/dotfiles/./config/sublime-text-3/Packages/User/GitHub.sublime-settings" VS "/home/vene/.config/sublime-text-3/Packages/User/GitHub.sublime-settings"
5c5
<             "github_token": ""
---
Overwrite "/home/vene/.config/sublime-text-3/Packages/User/GitHub.sublime-settings" [y/N] ? 
[WARN] ignoring /home/vene/.config/sublime-text-3/Packages/User/GitHub.sublime-settings 
[WARN] ignoring /home/vene/.config/git/config 
diff "/home/vene/.dotfiles/dotfiles/./config/git/credentials" VS "/home/vene/.config/git/credentials"
Binary files /tmp/dotdrop-c9fp0to9 and /home/vene/.config/git/credentials differ
Overwrite "/home/vene/.config/git/credentials" [y/N] ? 
[WARN] ignoring /home/vene/.config/git/credentials 

2 dotfile(s) installed.

[deadc0de6]

Awesome, I'll clean the code and add some doc and will merge that into master soon. Thanks for your help!

[deadc0de6]

I have merged this into the master branch. The config entry has been updated to import_variables. The doc is available here.