Invalid docs on mouting docker configs

[th0ger]

What would you like to be documented:

Mount a Config File to Each Node says:

If you pre-create a docker config.json containing credential(s) on the host you can mount it to each kind node.

Assuming your file is at /path/to/my/secret.json, the kind config would be:

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
  extraMounts:
  - containerPath: /var/lib/kubelet/config.json
    hostPath: /path/to/my/secret.json

But if I do this with my hostPath: ~/.docker/config/secret.json, it mounts an empty directory /var/lib/kubelet/config.json/.

The extra mounts docs only states how to mount directories. I assume that extra mount are not able to mount single files. But which directory should I mount the .docker/config.json to then?

Why is this needed:

It's not clear how/where to mount docker secrets.

[th0ger]

I found out that

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
  extraMounts:
  - containerPath: /var/lib/kubelet/.docker/config.json
    hostPath: ~/.docker/config.json

creates a local folder structure in the pwd where I call kind create cluster --config=config.yaml: /home/USER_NAME/PWD_DIR/~/.docker/config.json/. Yes, a literal \~ dir structure, super trippy side effect.

[BenTheElder]

"~" is not a real path character. You should expand that value (your shell will expand it to the real path for $HOME) or this is a feature request to specially handle that character.

[th0ger]

@BenTheElder next I tried with with absolute path but couldn't get it to work. Is mounting a single file supported?

[BenTheElder]

I think you have the wrong path? (on the nodes: /var/lib/kubelet/.docker/config.json should be /var/lib/kubelet/config.json, this is also less portable than other options like imagepullsecret)

https://kind.sigs.k8s.io/docs/user/private-registries/

[th0ger]

Thanks @BenTheElder it's working with:

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
  extraMounts:
  - containerPath: /var/lib/kubelet/config.json
    hostPath: /home/[USER]/.docker/config.json  # Full path, don't attempt ~ or $HOME !

So the documentation is okay 👍🏼

I still find it odd that if you attempy to use ~/ or $HOME/ it will literally create the folders "~" or "$HOME" in the current workdir. It's totally fair if variable expansion are not supported. But trying to mount non-existing path A to B with the side effect of creating A (in addition to B) is really odd. If this is correct, I would call it a bug.

[BenTheElder]

I still find it odd that if you attempy to use ~/ or $HOME/ it will literally create the folders "~" or "$HOME" in the current workdir. It's totally fair if variable expansion are not supported.

These are valid file paths, when you're using a shell it treats ~ and $ specially, but in literal filepaths they're permitted (if unusual). These are literal filepaths.

But trying to mount non-existing path A to B with the side effect of creating A (in addition to B) is really odd. If this is correct, I would call it a bug.

This is the behavior of docker volumes, kind is not creating any directories / files.

[BenTheElder]

It would probably be a better user experience to enable automatically expanding these, but it would also be a breaking change, and things like discovering ~ lookup/expansion are subtly non-portable. We could consider adding this in a future release.

For creating the directory if it doesn't exist ... this is expected with docker / podman. And kind can't check and warn because the paths may be against a remote daemon.

[BenTheElder]

kind did add in the last api revision that ./ is resolved relative to the current invocation (IE the working directory of the kind CLI process), so you can use that for test configs (which is also more portable than depending on the user's $HOME).

[th0ger]

@BenTheElder thanks for explaining, I understand that kind can't do much then.

For creating the directory if it doesn't exist ... this is expected with docker / podman.

Yes, the target dir I expected but the source dir also... Do you by chance have a ref. for that? (Food for thought: cp A B == mkdir A B.)

[stmcginnis]

I'm not sure about docs, but:

$ tree foo
foo  [error opening dir]

0 directories, 0 files

$ docker run -v ./foo/bar:/foo/bar bash tree /foo
/foo
└── bar

1 directories, 0 files

$ tree ./foo
foo
└── bar

1 directory, 0 files

[BenTheElder]

If you think about the mount as a bidirectional sharing from the container and the host it sort of makes sense to just ensure both ends exist.

I don't have a reference handy, but you can try something like docker run -v ./foo:./bar ubuntu ls to see it.

kind does not have any code to touch the host filesystem itself excepting:

• kind export kubeconfig (and the associated behavior in kind create cluster)
• kind delete cluster (clean up the kubeconfig)
• kind load docker-image (temp dir for images exported from docker)

When we add one of these mounts we're just passing the arguments to docker/podman/nerdctl in a structured way and the behavior of mounting them is in the container runtime. Which is defacto defined by docker.

[th0ger]

@BenTheElder for bidirectional mounts I think one could argue both. (By bidirectional you mean rw mode i presume?) For example, you can't make a symlink to a non-existent target. But it's still "bidirectional"/rw. But that's how it is then.

Anyway, I didn't realize from the kind docs that 1) this was bidirectional and 2) the side effect of creating the missing source. A note could be added to the extraMounts section if you prefer, but closing here.

Thanks for helping out.