Charm tutorials cannot use multipass blueprints reliably

[taurus-forever]

Hi,

The Charmed MySQL and PostgreSQL documentations use blueprint charm-dev. Manuals were written in Juju 2.9 times. Recently Juju latest/stable switched to Juju 3.1 and tutorial is now broken (note: charms support Juju 3.1). Issues on multipass launch:

* folder /home/ubuntu/.local/share needs to be created manually
* juju 3.1 requires microk8s-strict versions

At the moment there are PR to migrate charm-dev on Juju 3.2. Nice! Unfortunately Juju 3.2 was never tested by Data team => impossible to guaranties exceptional user experience in documentation. :-(

There were idea to create independent blueprints https://github.com/canonical/multipass-blueprints/pull/27 but it was rejected. Is it possible to reconsider it? Charm developers will be able to produce reliable manuals (by using immutable charm-dev-juju-3.1). The charm-dev can points to the latest Juju/LXD/MicroK8s/... always. Thank you!

P.S. Better ideas are welcome!

[townsend2010]

Hi @taurus-forever!

Once idea I have is perhaps we can have 2 charm-dev-related Blueprints. Once follows whatever is in latest/stable and then one for some development version that the charm-dev Blueprint author deems they want to follow, be it Juju 3.2 or newer.

We don't really have a story of multi-track Blueprints and having multiple related Blueprints can be confusing to an end user and not provide a good experience. I think 2 is manageable for this though.

@sed-i, since you are the main charm-dev Blueprint author, WDYT?

[sed-i]

Can we enable decoupling cloud-init from blueprints?

I think we would gain a lot if we increase separation of concerns —

multipass launch is already able to take cloud-init from URL:

multipass launch \
  --cloud-init https://raw.githubusercontent.com/Abuelodelanada/charm-dev-utils/main/cloud-init/charm-dev-juju-3.0.yaml \
  --name marc --mem 7G --cpus 2 --disk 50G \
  --mount ~/code:/home/ubuntu/code

and blueprints are excellent for provisioning multiple instances:

instances:
  microk8s-cluster-node-1:
    image: 22.04
    limits:
      min-cpu: 2
      min-mem: 4G
      min-disk: 30G
    timeout: 1200
    cloud-init:
      vendor-data: |
        # ...

I think that if we could:

1. Launch an instance from blueprint, and specify cloud-init url at the same time, instead of inlining it; and
2. Have a central store of cloud-init script per juju minor version

then we would benefit the most from each, and all our problems would be solved™.

[townsend2010]

Hi @sed-i!

Thanks for your feedback. It kind of does this already. If you launch charm-dev and pass it --cloud-init https://raw.githubusercontent.com/Abuelodelanada/charm-dev-utils/main/cloud-init/charm-dev-juju-3.2.yaml, it will provision the instance in the way the Blueprint defines, but runs the cloud-init defined in that URL and overrides any cloud-init defined in the Blueprint.

What I think would be best would be some way to merge what's in the Blueprint with what is in the passed in cloud-init. Where there are duplicate entries, the passed in cloud-init would have precedence. This way, a base cloud-init can be defined in the Blueprint with customizations added in the passed in cloud-init.

[sed-i]

My impression was that each instance in a blueprint could have a different cloud-init.

[taurus-forever]

Dear @townsend2010 , thank you for the attention here!

Can you share the working example for Juju 2.9 LTS, Juju 3.1 and Juju 3.2 blueprints?

P.S. I believe Canonical should advertise something internal and trusted: e.g. multipass launch juju-3.1 (like mulitpass launch 22.04) and not raw.githubusercontent.com/someone/something.yaml.

---

re: proposals above.

I agree, a lot of flexibility is available already, but simplicity is missing. The best documentation should looks like:

multipass launch -s juju-3.1

juju deploy mysql-k8s
juju deploy wordpress-k8s
juju integrate mysql-k8s wordpress-k8s
juju expose wordpress-k8s

(where -s will open a shell after the launch (separate topic). To remove necessity for multipass shell juju-3.1 after the multipass launch juju 3.1)

The proposed manual is hard to read (and support):

multipass launch \
  --cloud-init https://raw.githubusercontent.com/Abuelodelanada/charm-dev-utils/main/cloud-init/charm-dev-juju-3.0.yaml \
 ...

Thank you in advance!

[townsend2010]

My impression was that each instance in a blueprint could have a different cloud-init.

Hi @sed-i, we don't yet support creating multiple instances from a single Blueprint and the design for that has been in quite a bit of flux and I'm not sure when that support will happen.

[townsend2010]

Hi @taurus-forever!

Can you share the working example for Juju 2.9 LTS, Juju 3.1 and Juju 3.2 blueprints?

Sorry, I'm not really sure what you mean here. There aren't any Blueprints tracking that, just this charm-dev Blueprint. Do you mean different launches with the passed in cloud-init for the different Juju versions?

I agree passing in the cloud-init is not a good experience and is meant for a stop-gap in the short term, but given that we are drafting our 24.04 Roadmap right now, I don't think we have any spare capacity to address any of this for the next 6 months.

Taking a step back from this, when we accepted the charm-dev Blueprint, the original intent was that it was going to be consumed by Canonical engineers doing Charm development and so rough edges were OK Since this seems to be changing and we want external developers using this to develop Charms, then I think we need a discussion with the stakeholders on how better we can service this. My point of view is that not only do I want a great user experience when using these Blueprints, I also want a good experience for the author of the Blueprints.

[sed-i]

Clarifying my proposal:

1. Keep cloud-init scripts under the multipass-blueprints repo

.
├── cloud-init
│   ├── juju-3.1.yaml
│   └── juju-3.2.yaml
└── v1
    ├── anbox-cloud-appliance.yaml
    └── charm-dev.yaml

2. Optionally, reference the cloud-init scripts from within blueprints

description: A microk8s-juju dev cluster

runs-on:
- x86_64

instances:
  node-1:
    cloud-init:
      from-file: ../cloud-init/juju-3.1.yaml
  node-2:
    cloud-init:
      from-file: ../cloud-init/juju-3.1.yaml

3. User can override cloud-init on launch

multipass launch blueprint_name \
  --cloud-init node1=juju-3.2.yaml \
  --cloud-init node2=juju-2.9.yaml

[townsend2010]

Hi @taurus-forever!

I've approved #34, so when that is merged, will you consider this issue resolved?

[taurus-forever]

will you consider this issue resolved?

The root of the issue is solved: DB manuals work again and based on Juju 3.1 (3.x LTS). Thank you!

The second topic: user experience (based on @sed-i proposals). It can be extracted, based on your comment we need stakeholder decision and planning (if we want to use blueprints in official manuals for end users). CC: @jnsgruk (as requester of multipass in our docs).

Thank you @townsend2010 for the qiuck merge of https://github.com/canonical/multipass-blueprints/pull/34!