MitoZ cannot access input files - Singularity on HPC

[PatrickCKennedy]

Hello,

Thank you for creating MitoZ.

I am trying to run MitoZ 3.6 via Singularity on an HPC cluster, using the following:

module load apps/singularity/1.1.3
this_wasp_name=PETIO_LR10_1
this_wasp_val_1_FILE=PETIO_LR10_1_CKDL220030777-1A_HK7J7DSX5_L4_1_val_1.fq.gz 
this_wasp_val_2_FILE=PETIO_LR10_1_CKDL220030777-1A_HK7J7DSX5_L4_2_val_2.fq.gz

singularity run --bind /user/work/pk14885/Input_wasps_for_Mitoz/$this_wasp_name:/mnt /user/work/pk14885/MitoZ_v3.6.sif mitoz all \
    --fq1 $this_wasp_val_1_FILE \
    --fq2 $this_wasp_val_2_FILE \
    --outprefix PETIO_LR10_1 \
    --genetic_code 5 \
    --clade Arthropoda \
    --fastq_read_length 150 \
    --data_size_for_mt_assembly 3,0 \
    --assembler megahit \
    --kmers_megahit 59 79 99 \
    --thread_number 1 \
    --memory 50 \
    --requiring_taxa Arthropoda

However, MitoZ cannot access my input files. It produces the following error:

INFO:    fuse2fs not found, will not be able to mount EXT3 filesystems
INFO:    underlay of /etc/localtime required more than 50 (70) bind mounts

2024-03-04 12:58:23,191 - mitoz.utility.utility - INFO - 
Found: /app/anaconda/envs/mitoz3.6/bin/spades.py

2024-03-04 12:58:23,191 - mitoz.utility.utility - INFO - 
Found: /app/anaconda/envs/mitoz3.6/bin/megahit

2024-03-04 12:58:23,191 - mitoz.utility.utility - INFO - 
all.main() got args:
Namespace(assembler='megahit', clade='Arthropoda', data_size_for_mt_assembly='3,0', debug=False, fastq_read_length='150', filter_by_taxa=True, filter_other_para='', fq1='PETIO_LR10_1_CKDL220030777-1A_HK7J7DSX5_L4_1_val_1.fq.gz', fq2='PETIO_LR10_1_CKDL220030777-1A_HK7J7DSX5_L4_2_val_2.fq.gz', func=<function main at 0x7fdffab70430>, genetic_code='5', insert_size='250', kmers=[71], kmers_megahit=['59', '79', '99'], kmers_spades=['auto'], logger=<Logger mitoz.utility.utility (DEBUG)>, memory='50', min_abundance='10', outprefix='PETIO_LR10_1', phred64=False, profiles_dir='/app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/profiles', requiring_relax='0', requiring_taxa='Arthropoda', resume_assembly=False, skip_filter=False, slow_search=False, species_name='Test sp.', template_sbt='/app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/annotate/script/template.sbt', thread_number='1', tmp_dir=None, workdir='./')

2024-03-04 12:58:23,211 - mitoz.utility.utility - ERROR - 
Can NOT access PETIO_LR10_1_CKDL220030777-1A_HK7J7DSX5_L4_1_val_1.fq.gz !!

Please may I ask if there is an obvious mistake I am making here?

[linzhi2013]

Probably here is the reason:

INFO:    fuse2fs not found, will not be able to mount EXT3 filesystems

It failed to mount, so that your --bind option did not work.

Please try to "shell" into the container and see if that works.

[PatrickCKennedy]

Thanks for the speedy reply.

When I run the following...

singularity shell --cleanenv \
    --bind /user/work/pk14885/Input_wasps_for_Mitoz/$this_wasp_name:/mnt \
    /user/work/pk14885/MitoZ_v3.6.sif

...it takes me into the Apptainer shell:

Apptainer> mitoz all -h
bash: mitoz: command not found

I also tried the following (after https://github.com/linzhi2013/MitoZ/issues/99#issuecomment-757007509):

Apptainer> /app/anaconda/bin/python3 /app/release_MitoZ_v3.6/MitoZ.py -h
/app/anaconda/bin/python3: can't open file '/app/release_MitoZ_v3.6/MitoZ.py': [Errno 2] No such file or directory

No luck so far!

[PatrickCKennedy]

Solved - it was an issue with how I was mounting the data to Singularity.

I was using:

--bind /user/work/pk14885/Input_wasps_for_Mitoz/$this_wasp_name:/mnt

That didn't allow MitoZ to access the data.

When I changed the bind option to remove the 'mnt' and replace it with the location of the dataset (in my case, /user/work/pk14885/Input_wasps_for_Mitoz/$this_wasp_to_sample/), it started running:

-B/user/work/pk14885/Input_wasps_for_Mitoz/$this_wasp_to_sample/:/user/work/pk14885/Input_wasps_for_Mitoz/$this_wasp_to_sample/

It still gives the fuse2fs warning before it gets going, but this doesn't seem to impede MitoZ.

Many thanks,

Patrick

[linzhi2013]

That's great!

Usually, I keep the values of the two sides of the --bind option the same, so it is easier to compose the command, i.e., you can use the "host path" inside the container.