Singularity Container read-only file system error

[pedrodcb]

Hello tbpore team,

Thank you for uploading the new tbpore 0.2.0 container. The new --db option seems to work without issue, however we're still getting this error upon running it:

  File "/usr/local/bin/tbpore", line 10, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.9/site-packages/tbpore/tbpore.py", line 506, in main
    main_cli()
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/tbpore/tbpore.py", line 258, in process
    tmp, logdir = setup_dirs(outdir, tmp)
  File "/usr/local/lib/python3.9/site-packages/tbpore/tbpore.py", line 110, in setup_dirs
    cache_dir.mkdir(parents=True, exist_ok=True)
  File "/usr/local/lib/python3.9/pathlib.py", line 1323, in mkdir
    self._accessor.mkdir(self, mode)
OSError: [Errno 30] Read-only file system: '/usr/local/lib/python3.9/site-packages/.cache'

That's not a section of the code we would be able to change ourselves, so could you please help us with it?

The command we're using is as follow: singularity exec -H path_to/tbpore tbpore_0.2.0.sif tbpore process --db data/decontamination_db/tbpore.remove_contam.fa.gz.map-ont.mmi -o sample_example/tbpore_out --cleanup sample_example/mada_1-7.subsampled.fastq.gz

Thanks again for your time and help!

[mbhall88]

So sorry again @pedrodcb. Once #44 is merged I'll make another new release that will fix this problem. Thank you so much for your patience and bug reports!

[mbhall88]

Okay, version 0.3.0 has just been added to bioconda. You should see a new container in a few hours I guess

[pedrodcb]

Hello @mbhall88, thank you very much for setting up the new container so quickly. We've tried it out and at first ran into this issue: Error: Invalid value for '-m' / '--metadata': Path '/usr/local/lib/python3.9/site-packages/data/decontamination_db/remove_contam.tsv.gz' does not exist.

Which we fixed by manually downloading the remove_contam.tsv.gz file into the data/decontamination_db folder within our workspace and adapting the command by adding the -m option as follows: singularity exec -H path_to/tbpore tbpore_0.3.0.sif tbpore process --db data/decontamination_db/tbpore.remove_contam.fa.gz.map-ont.mmi -m data/decontamination_db/remove_contam.tsv.gz -o sample_example/tbpore_out --cleanup sample_example/mada_1-7.subsampled.fastq.gz

However, we now run into the following error, which like the previous ones, we're unable to fix ourselves: Error calling /usr/local/bin/python3.9 /usr/local/lib/python3.9/site-packages/external_scripts/filter_contamination.py --verbose --ignore-secondary -o sample_example/tbpore_out/.tbpore/mada_1-7.decontaminated.filter -i sample_example/tbpore_out/.tbpore/mada_1-7.decontaminated.sorted.bam -m data/decontamination_db/remove_contam.tsv.gz (return code 2)

In the error log file: /usr/local/bin/python3.9: can't open file '/usr/local/lib/python3.9/site-packages/external_scripts/filter_contamination.py': [Errno 2] No such file or directory

Once again thank you for your time and help with this!

[mbhall88]

I am so sorry for this @pedrodcb. I will fix this today

[mbhall88]

Alright, I think this will be sorted in v0.3.1. The external_scripts and data directories were being included in the package distribution, but not when pip installs from those distributions. Being more explicit with the files we want included seemed to fix this problem when I tested it locally.

I'll wait until the bioconda container is built and then test that out though.

[mbhall88]

Okay, I have tested the v0.3.1 container and I've successfully run the example you were trying to run @pedrodcb. I'm very sorry, I should have tested it this way in the previous fixes.

Let me know if you run into any other issues. Thanks for you patience.