run error when it runs with apptainer (HPC)

[Norina28]

Hello, I was trying to run PGAP on the cluster (with this command: ./pgap.py -r -o file_annotation ./file/input.yaml --cpus 18 -D /usr/bin/apptainer --ignore-all-errors) using apptainer but I got an error:

PGAP version 2022-04-14.build6021 is up to date.
WARNING: apptainer version 1.0.3
 (/usr/bin/apptainer) support as Docker alternative has not been tested
Output will be placed in: /home/ecella/Assemblies_final/PGAP/file_annotation
Error for command “run”: unknown shorthand flag: ‘v’ in -v

Run ‘apptainer --help’ for more detailed usage information.
Command ‘[’/usr/bin/apptainer’, ‘run’, ‘-i’, ‘-v’, ‘/home/ecella/Assemblies_final/PGAP:/cwd’, ‘ncbi/pgap:2022-04-14.build6021’, ‘bash’, ‘-c’, ‘df -k /cwd /tmp ; ulimit -a ; cat /proc/{meminfo,cpuinfo}‘]’ returned non-zero exit status 1.

I also tried to remove that short-hand option but still the same error

Thanks in advance

Ele

[azat-badretdin]

Thank you for your report, Ele.

We currently do not support this container runner (apptainer).

Before we jump in and commit to the support of this new (for us) container runner, let me ask you:

Do you have a workaround in your setup? Can you use singularity directly?

[Norina28]

Thanks for your reply. Unfortunately, singularity was updated to apptainer now in our cluster and I do not have any other way to run tools that require docker.

[azat-badretdin]

Thanks, @Norina28 . RIght, I learned that too: singularity as a marketing name is dead, long live apptainer.

[azat-badretdin]

We will schedule a change in our code

[Norina28]

Thanks so much. Really really appreciate.

[azat-badretdin]

For the record:

Error for command “run”: unknown shorthand flag: ‘v’ in -v

happens because -D is not called with "singularity" in it and it resorts to default container runner

[azat-badretdin]

Hi, @Norina28

I prepared a modified version of the script in dev branch, feel free to try it by downloading it from https://github.com/ncbi/pgap/raw/dev/scripts/pgap.py and subsequently always adding --no-self-update flag to your ./pgap.py calls.

[Norina28]

Sorry for my late response but I was trying to troubleshoot. I got a new error. ./pgap.py -r -o NAS_AN_016_G1_annotation ./NAS_AN_016_G1/input.yaml --cpus 18 --no-self-update -D /usr/bin/apptainer

this is the error:

PGAP version 2022-04-14.build6021 is up to date.
Not trying to update self, because the --no-self-update flag is enabled.
Output will be placed in: /home/ecella/SA_CAR/LongReadsAssemblies/Assemblies_final/PGAP/NAS_AN_016_G1_annotation
INFO:   Converting OCI blobs to SIF format
FATAL:   Unable to handle docker://ncbi/pgap:2022-04-14.build6021 uri: while building SIF from layers: unable to create new build: while ensuring correct compression algorithm: while creating squashfs: create command failed: exit status 1: FATAL ERROR:Failed to create thread
Command ‘[’/usr/bin/singularity’, ‘exec’, ‘--bind’, ‘/home/ecella/SA_CAR/LongReadsAssemblies/Assemblies_final/PGAP:/cwd:ro’, ‘docker://ncbi/pgap:2022-04-14.build6021’, ‘bash’, ‘-c’, ‘df -k /cwd /tmp ; ulimit -a ; cat /proc/{meminfo,cpuinfo}‘]’ returned non-zero exit status 255.

Thanks so much!

[azat-badretdin]

Thank you, Norina28, for your report!

We will have a look locally

[azat-badretdin]

It still goes for singularity for some reason in y our case... Could you please post your ./pgap.py copy? Thanks!

[azat-badretdin]

Or better grep apptain pgap.py

Does it mention apptainer?

[Norina28]

yes it is.

if self.params.docker_type in ['singularity', 'apptainer']:
        if (self.params.docker_type in ['singularity', 'apptainer']):
        docker_type_alternatives = ['docker', 'podman', 'singularity', 'apptainer']
        if self.docker_type in ['singularity', 'apptainer']:
                        help='Docker-compatible executable (e.g. docker, podman, apptainer), which may include a full path like /usr/bin/docker')

[azat-badretdin]

Then I can't explain it. It works on our AWS instance where I installed it.

Also, you have this "creating squashfs". Seems like you have a local system problem that needs to be resolved with your systems.

[Norina28]

Okay, thanks so much. I'll try to fix it locally in my cluster.