Sort error in running the CAMPHORsomatic

[SQLiu-youyou]

Hi, I've used your CAMPHORsomatic to detect TSSV in COLO829. But when I run, error happened:

sort: write failed: /tmp/sortYU0f0I: No space left on device

I thought it was occurred because the module sort would put the temporary file into the temporary directory which I didn't have much space for it. Thus, I edit your .sh file, and specifically gave the path to the sort module for the temporary directory producing. However, the same error happened again, which made me very confused. I really don't know how to do now. I hope you can help me.

Thanks,

Shiqi

[afujimoto]

Thank you for your interest. I think that your modification should resolve the problem. Did you check the disk space of your computer? Could you please tell me the shell script file name and line number of the error?

Regards,

Akihiro

2021/10/25 22:45、SQLiu-youyou @.***>のメール:

Hi, I've used your CAMPHORsomatic to detect TSSV in COLO829. But when I run, error happened:

sort: write failed: /tmp/sortYU0f0I: No space left on device I thought it was occurred because the module sort would put the temporary file into the temporary directory which I didn't have much space for it. Thus, I edit your .sh file, and specifically gave the path to the sort module for the temporary directory producing. However, the same error happened again, which made me very confused. I really don't know how to do now. I hope you can help me.

Thanks,

Shiqi

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/afujimoto/CAMPHORsomatic/issues/4, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADFCBLUY7KBDIOCCKXDIYNDUIVNQVANCNFSM5GVKPP6A. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[SQLiu-youyou]

Hi, Akihiro Thank you for your reply! I checked my disk space before running the CAMPHORsomatic, and I'm sure the space is abundant about 300G. And this is the error information occurred last time, I think the error was happened in the CAMPHOR_SVcall.sh.

Thanks,

Shiqi

[afujimoto]

Dear Shiqi,

If I have the asme problem, I will try to use non-tmp directory. Changing the directory name from -T /tmp/sortYU0f0I to -T /AAA would save the intermediate files to AAA directory. (In our system, tmp directory has the limitation of data amount)

Please let me know if you have problems after the modification. If you send me the modified CAMPHOR_SVcall.sh, I will take a look at the code.

Regards,

Akihiro

2021/10/26 17:40、SQLiu-youyou @.***>のメール:

Hi, Akihiro Thank you for your reply! I checked my disk space before running the CAMPHORsomatic, and I'm sure the space is abundant about 300G. https://user-images.githubusercontent.com/76677902/138841936-60071bf6-7427-4953-b93f-fd799ea0ee89.png And this is the error information occurred last time, I think the error was happened in the CAMPHOR_SVcall.sh.

Thanks,

Shiqi

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/afujimoto/CAMPHORsomatic/issues/4#issuecomment-951714992, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADFCBLRIOB2DWLTYAAXBN23UIZSPFANCNFSM5GVKPP6A. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[SQLiu-youyou]

Dear Akihiro,

Sorry to reply you so late, because I remapped the read using the new reference for a better performance. But problems still occurred during the running, and It seems like the same problems as before.

sort: write failed: /tmp/sortjZkdLe: No space left on device
Traceback (most recent call last):  File "./src/SV_support_read.py", line 706, in <module>
    print (min_distance_read_l[1], min_distance_read_l[2], min_distance_read_l[3], min_distance_read_l[4], str(len(breakpoint_tmp[2]) + num_l_BP + num_r_BP), "INS", str(breakpoint_tmp[1]), BP_num, read_name_all, sep="\t") #min_distance_read_l[0] => read_name_all
BrokenPipeError: [Errno 32] Broken pipe
sort: write failed: /tmp/sortf4ZYVz: No space left on deviceTraceback (most recent call last):
  File "./src/merge_BP.py", line 210, in <module>    print(BP)
BrokenPipeError: [Errno 32] Broken pipe

But I didn't see the /tmp/sortjZkdLe in neither the .sh nor in my storage. And the program didn't exit but still running after the BrokenPipeError. After the modification, there are 3 file in the sort_temp(the path I give to the sort module) :

65G sort7ugcid
65G sortpobeSH
43G sortqNPHgE

And I uploaded the modified CAMPHOR_SVcall.sh on my github https://github.com/SQLiu-youyou/Revise/blob/main/modified.SVcalling, and if you think this is inappropriate, I'll delete it immediately. I wish you can take a look at it.

Thanks,

Shiqi

[afujimoto]

Dear Shiqi,

I modified code in https://github.com/SQLiu-youyou/Revise/edit/main/modified.SVcalling https://github.com/SQLiu-youyou/Revise/edit/main/modified.SVcalling

I changed "python3 $SRC/SV_candidate.py $OUTPUT/SV_read.txt|sort -k2,2g -k3,3 -k4,4g -T /storage/sqliu/somatic/sort_temp|perl $SRC/SV_format_change.5.pl /dev/stdin > $OUTPUT/SV_read.txt.2"

to " python3 $SRC/SV_candidate.py $OUTPUT/SV_read.txt > $OUTPUT/SV_read.txt.1 sort -k2,2g -k3,3 -k4,4g -T /storage/sqliu/somatic/sort_temp $OUTPUT/SV_read.txt.1 > $OUTPUT/SV_read.txt.1.sort perl $SRC/SV_format_change.5.pl $OUTPUT/SV_read.txt.1.sort > $OUTPUT/SV_read.txt.2 “

If my codes have problems, python3 and perl (1st and 3rd lines) should cause problems.

Could you please run the code? Sorry for your convenience.

Regards,

Akihiro

2021/11/01 11:10、SQLiu-youyou @.***>のメール:

Dear Akihiro,

Sorry to reply you so late, because I remapped the read using the new reference for a better performance. But problems still occurred during the running, and It seems like the same problems as before.

sort: write failed: /tmp/sortjZkdLe: No space left on device Traceback (most recent call last): File "./src/SV_support_read.py", line 706, in print (min_distance_read_l[1], min_distance_read_l[2], min_distance_read_l[3], min_distance_read_l[4], str(len(breakpoint_tmp[2]) + num_l_BP + num_r_BP), "INS", str(breakpoint_tmp[1]), BP_num, read_name_all, sep="\t") #min_distance_read_l[0] => read_name_all BrokenPipeError: [Errno 32] Broken pipe sort: write failed: /tmp/sortf4ZYVz: No space left on deviceTraceback (most recent call last): File "./src/merge_BP.py", line 210, in print(BP) BrokenPipeError: [Errno 32] Broken pipe But I didn't see the /tmp/sortjZkdLe in neither the .sh nor in my storage. And the program didn't exit but still running after the BrokenPipeError. After the modification, there are 3 file in the sort_temp(the path I give to the sort module) :

65G sort7ugcid 65G sortpobeSH 43G sortqNPHgE And I uploaded the modified CAMPHOR_SVcall.sh on my github https://github.com/SQLiu-youyou/Revise/blob/main/modified.SVcalling <x-msg://45/url>, and if you think this is inappropriate, I'll delete it immediately. I wish you can take a look at it.

Thanks,

Shiqi

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/afujimoto/CAMPHORsomatic/issues/4#issuecomment-955867512, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADFCBLQUXVU6ULP2CHNXKDLUJXZJPANCNFSM5GVKPP6A. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[SQLiu-youyou]

Dear Akihiro,

After your modification, the only error through the running is the "no space" in my computer. After cleaning my storage, I'll try it again. Thank you very much for giving me the help! By the way, I would like to know how much storage space is enough (I remember there is 600G left in my storage). Thanks again!

Thanks,

Shiqi

[afujimoto]

Dear Shiqi,

I am sorry for the delay in responding. I do not know how much disk space is needed. I recommend you use a machine with larger disk space or delete other files.

One idea is adding bzip2 to the file. For example, change from “$SAMTOOLS view $BAM|python3 $SRC/read_format.py /dev/stdin $MIN_INDEL_LENGTH 0.3|python3 $SRC/SV_format.py /dev/stdin 0.05 $MIN_INDEL_LENGTH 0 10 0.3 1000 20 > $OUTPUT/SV_read.txt” to “$SAMTOOLS view $BAM|python3 $SRC/read_format.py /dev/stdin $MIN_INDEL_LENGTH 0.3|python3 $SRC/SV_format.py /dev/stdin 0.05 $MIN_INDEL_LENGTH 0 10 0.3 1000 20 | bzip2 > $OUTPUT/SV_read.txt.bz2”

I slightly changed the “https://github.com/SQLiu-youyou/Revise/edit/main/modified.SVcalling https://github.com/SQLiu-youyou/Revise/edit/main/modified.SVcalling”. The change would reduce the size of the biggest file.

Please try to use this script. I hope this change helps you.

Regards,

Akihiro

2021/11/02 10:20、SQLiu-youyou @.***>のメール:

Dear Akihiro,

After your modification, the only error through the running is the "no space" in my computer. After cleaning my storage, I'll try it again. Thank you very much for giving me the help! By the way, I would like to know how much storage space is enough (I remember there is 600G left in my storage). Thanks again!

Thanks,

Shiqi

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/afujimoto/CAMPHORsomatic/issues/4#issuecomment-957016421, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADFCBLWJ5UIFFJOSZDEUZOTUJ44GLANCNFSM5GVKPP6A. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.