search

samtools / htsjdk

A Java API for high-throughput sequencing data (HTS) formats.
http://samtools.github.io/htsjdk/
276 stars 244 forks source link

htsjdk.samtools.util.IOUtil.assertFileIsWritable on certain NFS environments fails to correctly assess permissions #1671

Open jd-daniels opened 1 year ago

jd-daniels commented 1 year ago

Description of the issue:

When calling assertFilesIsWritable through htsjdk.samtools, through Picard mark duplicates, we find that IOUtil.java fails to correctly assert writable on NFS file shares that have special backend permissions. This NFS location is on Dell Powerscale and has ACL permissions to allow for multi protocol access. The user can write to the parent directory.

Exception in thread "main" htsjdk.samtools.SAMException: Cannot write file: . File does not exist and parent directory is not writable.. at htsjdk.samtools.util.IOUtil.assertFileIsWritable(IOUtil.java:562) at picard.sam.markduplicates.MarkDuplicates.doWork(MarkDuplicates.java:251) at picard.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:308) at picard.cmdline.PicardCommandLine.instanceMain(PicardCommandLine.java:103) at picard.cmdline.PicardCommandLine.main(PicardCommandLine.java:113)

Your environment:

Steps to reproduce

Run Picard mark duplicates on an NFS location that has multi protocol ACLs or probably any server side permission modifications like that. There is probably a simpler test case using that specific htsjdk.samtools.util.IOUtil.assertFileIsWritable.

Expected behaviour

The directory is writable to the user, so it should not be telling us that it isn’t writable. We use a lot of different software suites on this same NFS storage location that don’t seem to have an issue, so while a workaround is to use a NFS-only standard permission NFS share, it’s not ideal for many of our researchers..

Actual behaviour

File does not exist and parent directory is not writable..

lbergelson commented 1 year ago

@jd-daniels This is tricky for me to debug because I don't have access to a similar system.

It's strange that the reported file path is blank. (Cannot write file: .). It means that for some reason file.getAbsolutePath() is returning an empty string. Can you include an example command line that fails here? Does providing an absolute path vs a relative one make any difference? It looks like it's failing with the metrics file path specifically so we should check if there is anything strange about that in particular.

I doubt it will fix this problem, but you might want to try upgrading your picard version since that one is pretty old.

michaeljmetzger commented 2 weeks ago

We are having this exact error. Was there ever a fix for it?