Add podman play kube --build support for podman remotes

[1player]

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind feature

Description

podman play kube --build is not supported on remote podman. The documentation doesn't specify why, and I wasn't able to find any discussion of why exactly it isn't working, as podman build works on remote mode.

The use case is developing on immutable distributions like Fedora Silverblue. I'm running podman inside a toolbox, I can build images manually with podman --remote build, I would like podman play kube to be able to build as well, so I only need one command to local development.

Steps to reproduce the issue:

1. Set up a toolbox with access to the host podman socket
2. podman play kube --build

Describe the results you received:

Error: unknown flag: --build

Describe the results you expected:

Build to work on remote podman.

Additional information you deem important (e.g. issue happens only occasionally):

Output of podman version:

Client:       Podman Engine
Version:      4.1.0
API Version:  4.1.0
Go Version:   go1.18.1
Git Commit:   e4b03902052294d4f342a185bb54702ed5bed8b1
Built:        Fri May  6 19:18:30 2022
OS/Arch:      linux/amd64

Server:       Podman Engine
Version:      4.1.0
API Version:  4.1.0
Go Version:   go1.18
Built:        Fri May  6 17:15:54 2022
OS/Arch:      linux/amd64

[Luap99]

@baude PTAL

The problem with remote support is that remote build has to tar the full context dir and send it to the remote server. Currently the client only sends the yaml content to the server and the server does all the work. To make remote build work the client would need to parse the yaml and then build the images. This is significant more complex.

[github-actions[bot]]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

@flouthoc PTAL

[axel7083]

The problem with remote support is that remote build has to tar the full context dir and send it to the remote server.

Why is it a problem ? It is what podman build is doing isn't it ? One possibility could be to have --build default to false on remote machine, so the context-dir is not send if it is not explicitly required

[Luap99]

The client doesn't parse the yaml so it has no idea what is specified in the file as such it can never perform any builds.

You would need a major rewrite of the API in order to allow such things and this is not trivial.

[axel7083]

The client doesn't parse the yaml so it has no idea what is specified in the file as such it can never perform any builds.

@Luap99 After looking a bit at the code, I still think that the client does not need to parse the file.

build mechanism

Let's look at the build command and how it provide the context to the server. The client never parse the Containerfile, it creates a tar from the ContextDirectory (and other elements such as secrets).

https://github.com/containers/podman/blob/acc78af0c3870021ef683b8ca672522b8478b4d1/pkg/bindings/images/build.go#L438

Then it provide in the POST request at /build the tar file as the body

https://github.com/containers/podman/blob/acc78af0c3870021ef683b8ca672522b8478b4d1/pkg/bindings/images/build.go#L590

On the server side, it will ensure the content body is x-tar.

https://github.com/containers/podman/blob/bd00c6fef9b8dce1784fd531a8f3037eafb69008/pkg/api/handlers/compat/images_build.go#L42-L43

Then untar, and uses it as the ContextDirectory

https://github.com/containers/podman/blob/bd00c6fef9b8dce1784fd531a8f3037eafb69008/pkg/api/handlers/compat/images_build.go#L74C27-L74C41

kube play mechanism

When using kube play according to the documentation, the default ContextDirectory will be the parent folder of the yaml file provided. Or the value defined by --context-dir option.

The body of the request kube play is not the context directory as it is not supported, but simply concat all the yaml resources

https://github.com/containers/podman/blob/72f1617facbb83fbc1e74583f62394dfc6142109/pkg/bindings/kube/kube.go#L70

and provide them as body.

https://github.com/containers/podman/blob/72f1617facbb83fbc1e74583f62394dfc6142109/pkg/bindings/kube/kube.go#L78

Question

My question is, why just like the build command, where the Containerfile is not parsed by the client, and the context is simply send to the server, the same could be done for the kube play

[Luap99]

From the docs:

Kube play is capable of building images on the fly given the correct directory layout and Containerfiles. This option is not available for remote clients, including Mac and Windows (excluding WSL2) machines, yet. Consider the following excerpt from a YAML file:

apiVersion: v1
kind: Pod
metadata:
...
spec:
  containers:
  - name: container
    image: foobar
...

If there is a directory named foobar in the current working directory with a file named Containerfile or Dockerfile, Podman kube play builds that image and name it foobar. An example directory structure for this example looks like:

|- mykubefiles
    |- myplayfile.yaml
    |- foobar
         |- Containerfile

The build considers foobar to be the context directory for the build NOT . If there is an image in local storage called foobar, the image is not built unless the --build flag is used. Use --build=false to completely disable builds.

---

So without reading the file you cannot know what the dir on the client because the context dir is based of the image name foobar. Also without reading the file we wouldn't even know how to name the final image.

[axel7083]

So without reading the file you cannot know what the dir on the client because the context dir is based of the image name foobar. Also without reading the file we wouldn't even know how to name the final image.

I do not see the problem of sending the entire directory where myplayfile.yaml is, and let the backend uses what it needs after parsing it, just like the build command is sending all the context-dir without knowing in advance what the Containerfile will used.

[Luap99]

Sending a (possible) large directory by default is very bad.

[axel7083]

Sending a (possible) large directory by default is very bad.

I understand the concerns, context directory can be very big on build too, but we are still sending them. IMO the sizing issue is the user responsibility: when building an image or a pod making a context directory that is okey to deal with is on the user part.

I still think that this would be manageable, and could be improve by having the build option to false by default. Having this feature working on more platforms by supporting the remote feel like it would have a lot of benefits, overlapping the few drawback already facing when using podman build.

[Luap99]

If you want to propose a specific way on how to do it then sure please do so here.

You can also join our cabal meetings to discus it there, next one on July 2 (https://hackmd.io/gQCfskDuRLm7iOsWgH2yrg?both). So feel free to add this topic there.