Closed lictw closed 9 months ago
I have my .gitconfig
in some base config/profile and I need to extend it with company related things (generally set other name/email for some path prefix), in different combinations. The main limitation here that I can't store customer data (even names) in my public dotfile repository, so it's separated projects and I should attach things that need to be patched like plugins.
dynvariables and import_variables is very good fit, so I will clone required repositories and will use them as separate dotdrop projects, while my main project will attach some related to it variables to patch things, but without merges (or ability in j2 to list all variables to find by some key all related to current template, like gitconfig_A, gitconfig_B) I can't create some "infinity" variable, where first say to add foo
, second to add baz
and the result is:
gitconfig_extends:
customer_A: |-
foo
customer_B: |-
baz
Hope this explanation is clear..)
Is vanilla j2 has a way to find/access variable by dynamic name? Ansible has some variables that stores all available variables like vars/hostvars[name]
that allow something like vars[NAME]
or for key in vars.keys()
, and this's not only a way to solve this issue, but even more.
Lets for cleanliness I understand that I want in general - I want an ultimate platform for ricing. I want to have common bases and patch them for current rice, patch from rice, like plugins, maybe I'm doing something wrong or overcomplicating(..), I have implemented a small PoC to help better understand my thought, what I was trying to achieve:
And in general it all works, I have achieved this, but it feels like everything is done on the edge. For ex. pluggable dotfile profile
only works when latest config (that we use in dotdrop -c
) located at same level, so all same dotfiles should be in single config/configs on single level, and be imported into config at same level, an implementation on variables merges or lookups will be more clearly. Also the result config config.yaml
(we can't use rices/A/config.yaml
directly cause of problem above) works with profile rice-A
, but not with associated to host profile HOST
cause of complicated import hierarchy.
Once again, I'm not saying that everything should work like this, I'm saying what I'm trying to do in general, maybe I'm doing something wrong! It also can look like I'm trying to reinvent source .profile.d/* in .profile
and that is. This is how it works now with power of stow
: base dotfile creates .profile and anything else can add that it needs into .profile.d
, also it has [[ -x ~/.profile-next ]] && exec ~/.profile-next
at the end for chaining (start x11 for ex.), and I can say that in general it looks clearly, but:
include configs/*
for future extending, so I thought about templating.Currently dotdrop allows to lookup environement variables using something like {{@@ env['MY_VAR'] @@}}
. I could provide the same pattern for all variables to allow lookups.
For example providing all variables under the keyword _vars
which should allow you to do {{@@ _vars['MY_VAR'] @@}}
. Would that help?
In context of merges - yes, I think, if we will can list all variables with _vars.keys()
, filter them and extract something with name in variable like _vars[item]
or even _vars[_dotfile_key ~ "_default"]
will enough for all things related to current topic.
Small future investigation based on PoC and above messages: I understand that if config B extends/includes config A, than config B can't use dotfiles of config A if the configs stored in different folders, so their dotfiles stored in different places, config B has access to dotfiles A, but tries to lookup them relative to himself, not relative to config A as they are, BUT, if this dotfiles are in profile defined in config A, than config B can use this profile and related dotfiles will be accessed, is it by design or a bug? should I create an issue?
Small future investigation based on PoC and above messages: I understand that if config B extends/includes config A, than config B can't use dotfiles of config A if the configs stored in different folders, so their dotfiles stored in different places, config B has access to dotfiles A, but tries to lookup them relative to himself, not relative to config A as they are, BUT, if this dotfiles are in profile defined in config A, than config B can use this profile and related dotfiles will be accessed, is it by design or a bug? should I create an issue?
No that should be working. Here's something you can do:
parent-config.yaml
config:
backup: true
create: true
dotpath: some-dotpath-dir
import_configs:
- ../somepath/child-config.yaml
dotfiles:
f_abc:
dst: ~/.abc
src: abc
profiles:
p0:
include:
- p1
dotfiles:
- f_abc
child-config.yaml
config:
backup: true
create: true
dotpath: some-other-dotpath-dir
dotfiles:
f_def:
dst: ~/def
src: def
profiles:
p1:
dotfiles:
- f_def
Is that what you are looking for?
Yep, I said that it works via profile. But:
profiles:
p0:
dotfiles:
- f_abc
- f_def
this will fail with error like can't find some-dotpath-dir/def
. So dotdrop allows to use this dotfile f_def
defined in other config, it see the definition, there isn't error like unknown dotfile
/you can't use dotfiles from child config
, but dotdrop looks for this dotfile in wrong place.
Ok, I took some time to go through your poc and here are a few ideas:
_vars
in template which would contain a dictionary of all available variablesinclude
(see this doc). Where the variable for the path would be overwritten depending on the environment you wantHere's an example of the kind of filters you can add:
def vars_contain(variables, pattern):
"""
return dict of all vars containing
"pattern" in their key
"""
found = {}
for k in variables.keys():
if k.contains(pattern):
found[k] = variables[k]
return found
def vars_contain_to_list(variables, pattern):
"""
return list of the values of all vars containing
"pattern" in their key
"""
found = []
for k in variables.keys():
if k.contains(pattern):
found.append(variables[k])
return found
I have the feeling it became complex due to the fact that you're not able (for valid reasons) to store all dotfiles/configs in a single repository but have to selectively add them. Since you can't actually store those elements in clear text, wouldn't transformations help?
Also the dotpath
(as well as other elements of the config file) can be templated (see this doc). So it should be possible to actually have a base config (with all the shared dotfiles) and have a to-be-templated config file where the dotpath
would be changed depending on the environment and thus point to different environement/directory (client-A
, client-B
) where those would have a similar layout so that you could select one by changing the dotpath
value.
@lictw I justed merged #406 which adds the dictionary _vars
in templates containing all available variables.
You can have an example of how that works here:
{%@@ for key in _vars @@%}
key:{{@@ key @@}},value:{{@@ _vars[key] @@}}
{%@@ endfor @@%}
@deadc0de6 I'm late, sorry, I saw the answer, I will digest all this soon and will continue the topic, a little not up to it.. Thanks a lot for the detailed answer! (:
Like in Ansible, it will simplify some advanced things.