Closed nkuperwasser closed 3 months ago
I've run some fairly absurd things through FlashFry (millions of FASTA sequences against entire genomes, etc) and found good results by just splitting up the input and parallelizing with either Snakemake or GNU parallel.
Okay, thank you for the suggestion! Will look into snakemake and start working with it. I apologize in advance if this question sounds a bit naive (it's trial by fire for me in terms of python,r,unix,etc etc), but if I parallelize, is there a general consensus for the memory flags for the java instances and the size of the corresponding fasta files?
Sincerely, NK
This is something that depends on your compute resources, input size, database size, etc. FlashFry already uses concurrency, I have just found that at some large input sizes for large databases with many off targets it can hang on trying to write out all the data. Maybe the author has done some benchmarking and can weigh in better on requirements
Okay..thank you! I've been trying to run variations on the machine (via ssh) and twice in a row, I get a "broken pipe" and logged out...I will keep on trying to reduce the fasta sizes....(trial by fire :) )
Okay..thank you! I've been trying to run variations on the machine (via ssh) and twice in a row, I get a "broken pipe" and logged out...I will keep on trying to reduce the fasta sizes....(trial by fire :) )
Try giving tmux a try...it will preserve your session in case your SSH pipe breaks. tmux tutorial
Haven't had a chance to properly use tmux...got a newer silicon chip mac and have had some issues trying to get things back into order with various conda/python and other associated issues.
Sounds promising, and will start to do that soon.
Thank you very much for the creation of this program, works great! I am starting to assemble some fairly large regions to run the discover module, and I was wondering if/how it would be possible to run with multiple threads/cores?
Thanks!!
Sincerely,
NK