KeyError when deploying bundles with blank strings as default config values

[pengale]

I filed this against core here, https://bugs.launchpad.net/juju/+bug/1651260, but I'm working on a workaround.

... it appears that we need to pass our config as "config-yaml" rather than "config"

[pengale]

Right now, I'm pretty stuck. Wrote the following note to the juju-dev mailing list, asking for help:

Hi All,

I'm currently working on getting python-libjuju to successfully deploy the landscape-dense-maas bundle. It fails, as outlined in https://bugs.launchpad.net/juju/+bug/1651260.

(python-libjuju is a Python client that talks to Juju's websocket API; I'm currently using it inside the matrix testing framework.)

The comments in that bug suggest that the root of the problem is that I'm trying to deploy a charm (haproxy) that has an empty string as a default value, but python-libjuju is using the legacy "config" param when it calls ApplicationDeploy in the api, rather than the new "config-yaml" param. This sounds simple to fix -- all I have to do is change an arg from "config" to "config-yaml", and everything should work!

One hitch: the "config" param expects a json object, which is what we get back when we make our initial call to the planner, but config-yaml expects a string with yaml in it (this is per the logic in deployApplication in juju/apiserver/application/application.go).

That also sounds simple. As we do elsewhere in python-libjuju when we want to pass a yaml blob to the API, I use Python's handy yaml library, do yaml.dump(config), where "config" is the json config object I got from the planner, and everything should work!

This is where I'm stuck. If I pass in such a string, the websocket API simple hangs, and stops talking to me. I don't even see any error messages in the logs on my controller :-/

Does anyone have any insight as to what I might be doing incorrectly? In Python3, yaml.dump will produce a utf-8 string by default. All of that will get serialized to json before being submitted over the websocket, though, so I don't think that it's an encoding issue. (Passing the bundle in as a yaml blob to the planner in the first place works.) The config object I get back from the planner isn't wrapped in an "options" key, but adding that key before dumping the config to a yaml string doesn't fix the problem -- I still see the hang.

Apologies for the length of the post. And thanks in advance for anything you can do to get me unstuck!

~ PeteVG

[pengale]

I think that I've got a fix. From the note that I wrote to the mailing list:

Hi All,

Thank you for all the advice. @Andrew Wilkins: turning up logging unstuck me. :-)

Details follow, for those interested:

I've been mis-stating the root of the problem. It's not a blank value in config.yaml, it's a blank value in the "options" for haproxy in the bundle.yaml. The good news is that the config-yaml param thing is definitely the right fix for this.

The bug in python-libjuju wasn't an esoteric encoding issue -- it was simply a formatting issue, plus the fact that python-libjuju isn't interpreting all the error messages it gets back from the websocket api as errors, so I wasn't seeing the error from the gui as a traceback; I caught the error after doing a more careful read-through of the logs. I've filed a ticket against python-libjuju (https://github.com/juju/python-libjuju/issues/65) covering the issue with missing errors.

The actual formatting issue is that the websocket api wants the value of config-yaml to be a yaml string with everything under the application name. In Python, this line fixes everything:

options = yaml.dump({application: options})

(@Jeff Pihach: I don't see the GUI doing this, but I may be missing where it does it later on, outside of the code in your PR.)

In any case, I'll be submitting that as a patch to python-libjuju in the near future.

Meanwhile, thanks again for all your help.

~ PeteVG

[pengale]

Fixed by https://github.com/juju/python-libjuju/pull/66