`lib.interpolate` function not fully plumbed through in postBuildSteps.*.environment

[antifuchs]

Using inputs.buildbot-nix.lib.interpolate with the postBuildSteps.*.environment option doesn't result in the call to util.Interpolate being made in the resulting configuration; instead, the nix config writes {"value": "%(prop:attr)s", "_type": "interpolate"} to the configuration file, which then fails the config test:

with the config file portion generated like this:

  "post_build_steps": [
    {
      "command": "/nix/store/6a496z3csn66d4qfrs8mkaxkycx3asvs-deploy-fly/bin/deploy-fly",
      "environment": {
        "BOINKOR_ATTR": {
          "_type": "interpolate",
          "value": "%(prop:attr)s"
        },
        "BOINKOR_BRANCH": {
          "_type": "interpolate",
          "value": "%(prop:branch)s"
        },
        "BOINKOR_FLYSCRIPT": {
          "_type": "interpolate",
          "value": "%(prop:out_path)s/bin/deploy-fly"
        }
      },
      "name": "Deploy to fly.io"
    }
  ],

This is the log message that happens on startup:

Sep 22 17:42:10 gloria twistd[398728]: 2024-09-22T17:42:10-0400 [-] error while parsing config file:
Sep 22 17:42:10 gloria twistd[398728]:         Traceback (most recent call last):
Sep 22 17:42:10 gloria twistd[398728]:           File "/nix/store/0bkcrjqrdkhwrvj2574jji8hmrq7mcsm-python3-3.12.5-env/lib/python3.12/site-packages/twisted/python/threadpool.py", line 285, in <lambda>
Sep 22 17:42:10 gloria twistd[398728]:             inContext.theWork = lambda: context.call(  # type: ignore[attr-defined]
Sep 22 17:42:10 gloria twistd[398728]:           File "/nix/store/0bkcrjqrdkhwrvj2574jji8hmrq7mcsm-python3-3.12.5-env/lib/python3.12/site-packages/twisted/python/context.py", line 117, in callWithContext
Sep 22 17:42:10 gloria twistd[398728]:             return self.currentContext().callWithContext(ctx, func, *args, **kw)
Sep 22 17:42:10 gloria twistd[398728]:           File "/nix/store/0bkcrjqrdkhwrvj2574jji8hmrq7mcsm-python3-3.12.5-env/lib/python3.12/site-packages/twisted/python/context.py", line 82, in callWithContext
Sep 22 17:42:10 gloria twistd[398728]:             return func(*args, **kw)
Sep 22 17:42:10 gloria twistd[398728]:           File "/nix/store/0bkcrjqrdkhwrvj2574jji8hmrq7mcsm-python3-3.12.5-env/lib/python3.12/site-packages/buildbot/config/master.py", line 123, in loadConfig
Sep 22 17:42:10 gloria twistd[398728]:             filename, config_dict = loadConfigDict(self.basedir, self.configFileName)
Sep 22 17:42:10 gloria twistd[398728]:         --- <exception caught here> ---
Sep 22 17:42:10 gloria twistd[398728]:           File "/nix/store/0bkcrjqrdkhwrvj2574jji8hmrq7mcsm-python3-3.12.5-env/lib/python3.12/site-packages/buildbot/config/master.py", line 85, in loadConfigDict
Sep 22 17:42:10 gloria twistd[398728]:             execfile(filename, localDict)
Sep 22 17:42:10 gloria twistd[398728]:           File "/nix/store/0bkcrjqrdkhwrvj2574jji8hmrq7mcsm-python3-3.12.5-env/lib/python3.12/site-packages/twisted/python/compat.py", line 213, in execfile
Sep 22 17:42:10 gloria twistd[398728]:             exec(code, globals, locals)
Sep 22 17:42:10 gloria twistd[398728]:           File "/nix/store/amgyb4xapysihya0mwrkdg8nw2wz2wxz-master.cfg", line 25, in <module>
Sep 22 17:42:10 gloria twistd[398728]:             BuildbotNixConfig.model_validate(json.loads(Path("/nix/store/w8j4klyvqvh2v1z3dfby82gn7qxr7jiz-buildbot-nix-config.json").read_text()))
Sep 22 17:42:10 gloria twistd[398728]:           File "/nix/store/0bkcrjqrdkhwrvj2574jji8hmrq7mcsm-python3-3.12.5-env/lib/python3.12/site-packages/pydantic/main.py", line 568, in model_validate
Sep 22 17:42:10 gloria twistd[398728]:             return cls.__pydantic_validator__.validate_python(
Sep 22 17:42:10 gloria twistd[398728]:         builtins.TypeError: Interpolate.__init__() got an unexpected keyword argument '_type'

I believe that happens because the interpolateToString function defined in master.nix isn't called on those environment values.

[MagicRB]

Not quite, interpolateToString is dead code and was missed during the pydantic refactor. The breakage you're experiencing is due to pydantic sucking a lot at field renames. See the following. The fix here is for me to give up and not use _type on the nix side but rather nix_type as both _type and type are reserved on the Python side, former by pydantic and latter by Python.

In fact this used to work, I'm sure of it so some update of our dependencies must have somehow broken this.

[MagicRB]

I'll create a PR probably tomorrow, or you can if I managed to explain myself correctly :)

[MagicRB]

Wait nevermind, I see it.I'll do a quick PR now

[MagicRB]

@antifuchs done, you can test the PR but it should work, I've tested it.