search

NERSC / podman-hpc

Other
35 stars 7 forks source link

Support podman's `--noout` flag #64

Open wholtz opened 1 year ago

wholtz commented 1 year ago

podman has a --noout flag that must come before the sub-command (pull):

$ podman --noout pull --quiet debian:bullseye-slim 
$

podman-hpc does not support this usage:

$ podman-hpc --noout pull --quiet debian:bullseye-slim
Usage: podman-hpc [options] COMMAND [ARGS]...
Try "podman-hpc --help" for help.

Error: no such option: --noout
$
lastephey commented 1 year ago

I agree that it would be nice to add some mechanism to handle the meta podman options like --version. I'm sure @danfulton (currently traveling) will have some thoughts about this.

In the meantime, does this work for you? Also podman run has a similar -q flag. 

stephey@perlmutter:login25:~> podman-hpc pull -q ubuntu:latest > /dev/null
stephey@perlmutter:login25:~> podman-hpc pull -q ubuntu:fake > /dev/null
Error: initializing source docker://ubuntu:fake: reading manifest fake in docker.io/library/ubuntu: manifest unknown: manifest unknown
Pull failed.
stephey@perlmutter:login25:~>
wholtz commented 1 year ago

Yes, that meets my needs. Thanks!

danfulton commented 1 year ago

I thought at one point these options were working. I wonder if some of the pass through options got broken at some point. I'll try to take a look at this.

lastephey commented 1 year ago

@danfulton any chance you can take a look at this sometime this week?

danfulton commented 1 year ago

Yeah, planning to look tomorrow!

mamelara commented 1 year ago

I agree that it would be nice to add some mechanism to handle the meta podman options like --version. I'm sure @danfulton (currently traveling) will have some thoughts about this.

In the meantime, does this work for you? Also podman run has a similar -q flag.

stephey@perlmutter:login25:~> podman-hpc pull -q ubuntu:latest > /dev/null
stephey@perlmutter:login25:~> podman-hpc pull -q ubuntu:fake > /dev/null
Error: initializing source docker://ubuntu:fake: reading manifest fake in docker.io/library/ubuntu: manifest unknown: manifest unknown
Pull failed.
stephey@perlmutter:login25:~>

I actually have a use case for this because I want to use podman-compose but they have some logic in their code that uses the --version flag to determine whether podman is running. What seems to be the issue with supplying this option?

danfulton commented 1 year ago

@mamelara is podman-compose not just calling podman --version directly? If so, there should be no issue with that.

Many of these tickets about command-line gotchas are symptoms of how we are passing through commands from podman-hpc to podman. The CLI tree for podman is quite extensive, and so we haven't directly implemented it in the podman-hpc wrapper, since it would be significant effort and difficult to maintain. Instead we try to detect podman-hpc specific syntax and pass the remaining command string to podman, but this is a non-trivial syntax parsing task. Hopefully that gives some clarity.

We do have a global solution in mind which should hopefully address all of these issues at once.

mamelara commented 1 year ago

So would it be okay to use podman directly on NERSC then? I was under the impression that we needed to use podman-hpc on Perlmutter. podman-compose allows you to point to the location of the podman executable so I had it pointing to /usr/bin/podman-hpc but that error'd out since --version is not a valid arg.

danfulton commented 1 year ago

Ah, I see. You probably do want the --version flag on podman-hpc then. You will find podman in your path, but it is not configured correctly to be used directly by users. There are additional features in podman-hpc which enable efficient use of containers for parallel batch jobs as well.