Generate certificates first such that they can be bind-mounted into containers

[jbemmel]

Use case: A setup with a Salt Master/Proxy container and 2 SRL nodes. The proxy needs access to the certificates generated for the SRL containers (gNMI)

It works when doing 'destroy' followed by 'deploy', but '--reconfigure' messes things up

[hellt]

I am not sure this can work with reconfigure, as it removes the lab directory completely where the keys are stored.

The option to use is a combo of destroy/deploy or introduce a redeploy command, that can take in a --cleanup flag that is when set will equal to deploy --reconfigure, but without it it will be just destroy + deploy.

Another option worth exploring is to create a config snippet that contains cert information and is mounted to srl nodes. At the same time this cert/key is created OOB once, and mounted to salt containers.

That way srl should use the config with an extra tls profile that it will use for gnmi and salt nodes would have access to key material.

But maybe the first question is why salt can't work with --skip-verify?

[jbemmel]

It's a Salt Nornir proxy, I'm trying to get it to make gNMI calls from a Napalm driver that doesn't support --skip-verify

[jbemmel]

I haven't looked in detail, but I suspect the issue is a timing issue: Certificates get generated during pre-deploy, for each container separately (in parallel): https://github.com/srl-labs/containerlab/blob/main/clab/clab.go#L219

If we would change that to a 2-step approach, i.e. first generate certificates for all nodes and then create them, it would work

[hellt]

It might still be reasonable to add --skip-verify support for the backend used in napalm for gnmi. Upstream pygnmi has it, if they use it.

Secondly, you don't really need keys to make TLS work, you only need a ca.cert file to verify the server keys. Unless you must use mutual verification, of course. So I think there might be an easy way out of this by following the same approach we did for ansible inventory file. We can create an empty ca.cert file before the nodes start and allow containers to mount this empty file with binds. Then once the TLS certificate gets created the empty file becomes a filled-in file so that salt can make use of it.

So the potential flow is:

• bind mount ${labdir}/ca/ca.cert to salt container. Initially it is created empty
• once the code reaches the state where CA is created, we open the file for writing and populate contents
• salt is now able to use that file as a ca.cert to verify TLS setup

[jbemmel]

I'm just thinking that there are cases where it can be useful to have access to generated certificate files, by bind-mounting them into the containers. The easiest, most generic solution is to first generate all certificates, and then start the containers (instead of generating them as part of the container startup process)