Hiera: Implement the lookup function (in yaml file)

[PierreR]

For instance:

application::profile::pgserver::common::pg_srvdir: "%{lookup('datadir')}/pgsql"

Ref: https://docs.puppet.com/puppet/5.2/hiera_interpolation.html#lookup--hiera

I initially thought this was not blocking but I have a situation where the value for such a key is validated by the nginx puppet module :-(

Path must be absolute, not %{lookup('datadir')}/pgsql for parameter path at # SourcePos {sourceName = "./modules/nginx/manifests/resource/vhost.pp", sourceLine = Pos 733, sourceColumn = Pos 3}%  

[bartavelle]

Alright, this requires quite a bit of rework here. This means the previous values must be known while parsing a given value :/

[bartavelle]

I just added a "yera" command that should help testing the Hiera part of the code individually.

[bartavelle]

I think I can implement that. However, it would be nice if you had time to check if it is possible to resolve variable between files, and if so, if there is an order for this.

[PierreR]

Do you mean in hiera in general ? If so the answer is yes. it is possible to declare in common.yaml a key such as datadir: '/srv' and then deeper in the hiera hierarchy to have a file pgserver.yaml with pg_srvdir: "%{lookup('datadir')}/pgsql".

I will check if it works both ways but I would assume it does given the documentation comment:

The value referenced by the lookup function can contain another call to lookup; if you accidentally make an infinite loop, Hiera detects it and fails instead of hanging indefinitely.

For the record, lookup/hiera only accepts string as argument but there is also an alias function that works with the other types such as hash, array and boolean.

[bartavelle]

Ah, this makes the problem much more annoying!

[PierreR]

I can confirm it works both ways. A lookup in hiera (using %lookup) will do a complete new lookup according to the hiera hierarchy.

[bartavelle]

Just pushed something, and did not break the tests (apparently). Can you tell me if it is a catastrophe? I do not have a good test environment right now.

[PierreR]

I am probably missing something but it does not seem to work. Trying to debug it with yera I don't seem to be able to make it work:

yera -c tests/hiera.yaml -q='datadir' ::operatingsystemmajrelease='7' ::zone='prod' ::hostgroup='middleware'
no match

hiera -c tests/hiera.yaml 'datadir' ::operatingsystemmajrelease='7' ::zone='prod' ::hostgroup='middleware'
/srv

[PierreR]

Oh I have found out my mistake ;-)

[bartavelle]

So, does it work? I did not test it whatsoever :)

[PierreR]

The yera command works but the lookup does not ... I have

yera -c tests/hiera.yaml -q 'datadir' ::operatingsystemmajrelease='7' ::zone='prod' ::hostgroup='middleware' ::subgroup='pgserver'
/srv

yera -c tests/hiera.yaml -q 'test' ::operatingsystemmajrelease='7' ::zone='prod' ::hostgroup='middleware' ::subgroup='pgserver'

where test is declared as

test: "%{lookup('datadir'}"

datadir is declared in hieradata/prod/middleware/common.yaml whereas test is declared in hieradata/prod/middleware/pgserver/common.yaml

[bartavelle]

isn't it a missing closing parens problem?

[PierreR]

Euh yes it is ... Sorry about that ;-)

[bartavelle]

So, does it work?

[PierreR]

Yes it seems to work fine. I go along with my testing. Thanks

[bartavelle]

Awesome! You will probably need to test it a bit, as I did not ;)