search

metageni / SUPER-FOCUS

A tool for agile functional analysis of shotgun metagenomic data
GNU General Public License v3.0
21 stars 12 forks source link

IsADirectoryError: [Errno 21] Is a directory: '/home/mdimdadi/14777R/FastQ/Passed_Reads/14777x5/5_superfocus_output' #49

Closed ghost closed 5 years ago

ghost commented 5 years ago

When I run the following command I get an error: $ superfocus -q 14777x5_All_Read_Merged.fastq -dir 5_superfocus_output/ -a diamond -t 25 -e 0.00001 -db DB_90 -l 5_superfocus_output/ &

[mdimdadi@rameshgoel 14777x5]$ Traceback (most recent call last): File "/usr/bin/superfocus", line 10, in sys.exit(main()) File "/usr/bin/miniconda3/lib/python3.6/site-packages/superfocus_app/superfocus.py", line 264, in main logging.basicConfig(format=LOGGER_FORMAT, level=logging.INFO, filename=args.log) File "/usr/bin/miniconda3/lib/python3.6/logging/init.py", line 1806, in basicConfig h = FileHandler(filename, mode) File "/usr/bin/miniconda3/lib/python3.6/logging/init.py", line 1030, in init StreamHandler.init(self, self._open()) File "/usr/bin/miniconda3/lib/python3.6/logging/init.py", line 1059, in _open return open(self.baseFilename, self.mode, encoding=self.encoding) IsADirectoryError: [Errno 21] Is a directory: '/home/mdimdadi/14777R/FastQ/Passed_Reads/14777x5/5_superfocus_output'

metageni commented 5 years ago

-l requires a file for the log rather than a directory.

ghost commented 5 years ago

Hi @metageni Thanks, that helped.

Now I ran the super-focus, it launches successfully. But after some time, I get the following error:

Processing query chunk 0, reference chunk 0, shape 1, index chunk 1. Building reference index... [5.16536s] Building query index... [4.31462s] Building seed filter... [0.37562s] Searching alignments... Error: Error writing file /usr/bin/miniconda3/lib/python 3.6/site-packages/superfocus_app/db/tmp/diamond-5cb18e66-15.tmp. Disk full? Error: terminate called after throwing an instance of 'File_write_exception' Error writing file /usr/bin/miniconda3/lib/python3.6/site-packages/superfocus_ap p/db/tmp/diamond-5cb18e66-21.tmp. Disk full? Error: Error writing file /usr/bin/miniconda3/lib/python3.6/site-packages/superf ocus_app/db/tmp/diamond-5cb18e66-20.tmp. Disk full? terminate called recursively what(): Error writing file /usr/bin/miniconda3/lib/python3.6/site-packages/su perfocus_app/db/tmp/diamond-5cb18e66-15.tmp. Disk full? terminate called recursively Error: Error writing file /usr/bin/miniconda3/lib/python3.6/site-packages/superf ocus_app/db/tmp/diamond-5cb18e66-49.tmp. Disk full? terminate called recursively Error: Error writing file /usr/bin/miniconda3/lib/python3.6/site-packages/superf ocus_app/db/tmp/diamond-5cb18e66-41.tmp. Disk full? terminate called recursively Error: Error writing file /usr/bin/miniconda3/lib/python3.6/site-packages/superf ocus_app/db/tmp/diamond-5cb18e66-45.tmp. Disk full? terminate called recursively Error: Error writing file /usr/bin/miniconda3/lib/python3.6/site-packages/superf ocus_app/db/tmp/diamond-5cb18e66-d.tmp. Disk full? terminate called recursively Error: Error writing file /usr/bin/miniconda3/lib/python3.6/site-packages/superf ocus_app/db/tmp/diamond-5cb18e66-2c.tmp. Disk full? terminate called recursively Error: Error writing file /usr/bin/miniconda3/lib/python3.6/site-packages/superf ocus_app/db/tmp/diamond-5cb18e66-9.tmp. Disk full? terminate called recursively Error: Error writing file /usr/bin/miniconda3/lib/python3.6/site-packages/superf ocus_app/db/tmp/diamond-5cb18e66-19.tmp. Disk full? terminate called recursively Error: Error writing file /usr/bin/miniconda3/lib/python3.6/site-packages/superf ocus_app/db/tmp/diamond-5cb18e66-3f.tmp. Disk full? terminate called recursively Error: Error writing file /usr/bin/miniconda3/lib/python3.6/site-packages/superf ocus_app/db/tmp/diamond-5cb18e66-11.tmp. Disk full? terminate called recursively Error: Error writing file /usr/bin/miniconda3/lib/python3.6/site-packages/superf ocus_app/db/tmp/diamond-5cb18e66-2d.tmp. Disk full? terminate called recursively Error: Error writing file /usr/bin/miniconda3/lib/python3.6/site-packages/superf ocus_app/db/tmp/diamond-5cb18e66-35.tmp. Disk full? terminate called recursively Error: Error writing file /usr/bin/miniconda3/lib/python3.6/site-packages/superf ocus_app/db/tmp/diamond-5cb18e66-39.tmp. Disk full? terminate called recursively Error: Error writing file /usr/bin/miniconda3/lib/python3.6/site-packages/superf ocus_app/db/tmp/diamond-5cb18e66-5.tmp. Disk full? terminate called recursively Error: Error writing file /usr/bin/miniconda3/lib/python3.6/site-packages/superf ocus_app/db/tmp/diamond-5cb18e66-1.tmp. Disk full? terminate called recursively Error: Error writing file /usr/bin/miniconda3/lib/python3.6/site-packages/superf ocus_app/db/tmp/diamond-5cb18e66-25.tmp. Disk full? terminate called recursively Error: Error writing file /usr/bin/miniconda3/lib/python3.6/site-packages/superf ocus_app/db/tmp/diamond-5cb18e66-31.tmp. Disk full? terminate called recursively Error: Error writing file /usr/bin/miniconda3/lib/python3.6/site-packages/superf ocus_app/db/tmp/diamond-5cb18e66-4d.tmp. Disk full? terminate called recursively diamond v0.8.22.84 | by Benjamin Buchfink buchfink@gmail.com Check http://github.com/bbuchfink/diamond for updates.

CPU threads: 32

Error: Invalid DAA file. DIAMOND run probably has not completed successfully. Traceback (most recent call last): File "/usr/bin/miniconda3/bin/superfocus", line 10, in sys.exit(main()) File "/usr/bin/miniconda3/lib/python3.6/site-packages/superfocus_app/superfocu s.py", line 342, in main del_alignments) ValueError: not enough values to unpack (expected 2, got 0)

Thanks again.

metageni commented 5 years ago

How much storage do you have available? The error message seems to point that the alignments temp file cannot be written Disk full?

ghost commented 5 years ago

[mdimdadi@rameshgoel data]$ df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/centos-root 50G 44G 6.8G 87% / devtmpfs 126G 0 126G 0% /dev tmpfs 126G 0 126G 0% /dev/shm tmpfs 126G 2.5G 124G 2% /run tmpfs 126G 0 126G 0% /sys/fs/cgroup /dev/sde1 497M 350M 147M 71% /boot /dev/mapper/centos-lv_local 59G 918M 56G 2% /usr/local /dev/mapper/vg_data-lv_shared 788G 275G 477G 37% /srv/data /dev/mapper/vg_data-lv_user 1.4T 894G 445G 67% /home tmpfs 26G 0 26G 0% /run/user/1001 tmpfs 26G 0 26G 0% /run/user/1012

can we divert to write the temp file to write in other place? disk_space

metageni commented 5 years ago

Try to modify https://github.com/metageni/SUPER-FOCUS/blob/master/superfocus_app/do_alignment.py#L77

I think it should be on your computer at /usr/bin/miniconda3/lib/python3.6/site-packages/superfocus_app/do_alignment.py

edit temp_folder to be the Path you want the temp files to go. I will add a fix on next release so people can pass it as an optional parameter.

ghost commented 5 years ago

when I open the do_alignment.py, I see temp_folder = Path("{}/db/tmp/".format(WORK_DIRECTORY))

I want to write the temp file say in /srv/data/ should I make it like the following:

temp_folder = Path("/srv/data/".format(WORK_DIRECTORY))

Thanks @metageni

metageni commented 5 years ago

Sure. Please keep me posted if it will solve your problem.

metageni commented 5 years ago

I think you will want to set temp_folder = PATH_WHERE_YOU_WANT_TEMP_FILES

temp_folder = Path("/srv/data/".format(WORK_DIRECTORY)) may not work.

ghost commented 5 years ago

so like this: temp_folder = Path("/srv/data/")

metageni commented 5 years ago

yep

ghost commented 5 years ago

The first way is working I think. It's not stopping anymore for DISK_FULL reason.

Thank you very much, You are awesome!!