Add support for reference bundles from repo URL

[johnsca]

Sometimes you want to test a charm within the context of a non-public bundle.

Relevant code: https://github.com/juju-solutions/layer-cwr/blob/master/actions/cwrhelpers.py#L76

[kwmonroe]

This isn't as simple as saying "gimme a repo and we'll just clone it."

Firstly, we're normalizing the bundle name to force it to start with cs:. This needs to be updated to allow http(s):// and maybe even a local:/// prefix:

https://github.com/juju-solutions/layer-cwr/blob/master/actions/cwrhelpers.py#L80

Second, we're using theblues to inspect the bundle.yaml directly from the charm store. This will need to be refactored to use a local bundle.yaml that we potentially downloaded/cloned:

https://github.com/juju-solutions/layer-cwr/blob/master/actions/cwrhelpers.py#L30

On the plus side, we do have a fetcher library that can handle all kinds of stuff (bzr, git, cs, local, etc):

https://github.com/juju/charm-tools/blob/master/charmtools/fetchers.py#L352

On the down side, this is only py2.

Still, we've agreed that the right approach is to generically fetch bundles so we can support a wide variety of sources... Then we'll store the yaml and share that location with any other method that needs to massage it. @johnsca suggested the fetching bits should happen in bundletester, and I agree. bt feels like the right place to do all-things bundle related, including fetching them.

Long story short, I don't see this being fixed in layer-cwr until we have the bundletester bits in place. For now, I'm afraid we'll insist that reference bundles must live somewhere in the charm store.

[kwmonroe]

There is a workaround for the cwr-bundle action that came in with #108:

https://github.com/juju-solutions/layer-cwr/pull/108/commits/2ca550c3b56b59740692920064c684ac0a8e21ce#diff-bcff72b1cbd554f1e10a6717db605fbaR61

Given a bundle is local on the cwr unit, you can run:

juju run-action cwr/0 cwr-bundle \
  repo=local:/path/to/bundle \
  bundle-name=awesome-bundle

This isn't pretty because it means you'd have to get a local bundle on the cwr unit in the first place, and this is only available for the cwr-bundle action. Still, I thought it worth mentioning.

@ktsakalozos as the author of #108, did you have a vision for addressing the limitations noted above (specifically, will this be supported in the other actions, and will we have a better UX for users getting a local bundle onto the cwr unit)?

[ktsakalozos]

The motivation behind https://github.com/juju-solutions/layer-cwr/pull/108 was slightly different. The Kubernetes team maintains a script that builds the bundle and keeps it locally.

Since we offer the option to specify the reference bundle as action parameter here we should also be able to fetch it from wherever the user wishes. That is not trivial but can be done. Pushing this functionality (fetch reference bundle from any location) to bundletester as well seems to be the right way to go. However, this would mean we would have to duplicate the fetcher logic. As @kwmonroe said https://github.com/juju/charm-tools/blob/master/charmtools/fetchers.py#L352 is a good place to start.

Does anyone remember who asked for this feature?

[johnsca]

I think it may have come from the Open Stack folks, because they have a lot of different configurations that they might want to test but not necessarily support as publicly available bundles.