Ray v1.7. Critial error found

[mscook]

Hi,

This may be a result of - 1) Poor cluster configuration 2) Stupidity on my part

BUT

Ray should halt immediately.

LOG:

Rank 6 is loading sequence reads Rank 6 : partition is [8133607;9489207], 1355601 sequence reads Rank 7 is loading sequence reads Rank 7 : partition is [9489208;10844808], 1355601 sequence reads Rank 1 is loading sequence reads Rank 1 : partition is [1355602;2711202], 1355601 sequence reads Rank 5 is loading sequence reads Rank 5 : partition is [6778006;8133606], 1355601 sequence reads Rank 3 is loading sequence reads Rank 3 : partition is [4066804;5422404], 1355601 sequence reads Rank 2 is loading sequence reads Rank 2 : partition is [2711203;4066803], 1355601 sequence reads Rank 4 is loading sequence reads Rank 4 : partition is [5422405;6778005], 1355601 sequence reads Rank 0 is loading sequence reads Rank 0 : partition is [1;1355601], 1355601 sequence reads Rank 9 is loading sequence reads Rank 9 : partition is [12200410;13556010], 1355601 sequence reads Ray: cannot access 'XXXXX_proc.fastq': No such file or directory Rank 9 has 0 sequence reads (completed) Rank 13 is loading sequence reads Rank 13 : partition is [17622814;18978414], 1355601 sequence reads Ray: cannot access 'XXXXX_proc.fastq': No such file or directory Rank 13 has 0 sequence reads (completed) Rank 10 is loading sequence reads Rank 10 : partition is [13556011;14911611], 1355601 sequence reads Ray: cannot access 'XXXXX_proc.fastq': No such file or directory Rank 10 has 0 sequence reads (completed) Rank 11 is loading sequence reads Rank 11 : partition is [14911612;16267212], 1355601 sequence reads Ray: cannot access 'XXXXX_proc.fastq': No such file or directory Rank 11 has 0 sequence reads (completed) Rank 12 is loading sequence reads Rank 12 : partition is [16267213;17622813], 1355601 sequence reads Ray: cannot access 'XXXXX_proc.fastq': No such file or directory Rank 12 has 0 sequence reads (completed) Rank 15 is loading sequence reads Rank 15 : partition is [20334016;21689626], 1355611 sequence reads Ray: cannot access 'XXXXX_proc.fastq': No such file or directory Rank 15 has 0 sequence reads (completed) Rank 8 is loading sequence reads Rank 8 : partition is [10844809;12200409], 1355601 sequence reads Ray: cannot access 'XXXXX_proc.fastq': No such file or directory Rank 8 has 0 sequence reads (completed) Rank 14 is loading sequence reads Rank 14 : partition is [18978415;20334015], 1355601 sequence reads Running on 16 Processors across 2 nodes. The second node does not seem to have access to the input file. Ray continues to run: OutputNumbers.txt: Contigs >= 100 nt Number: 218 Total length: 5424549 Average: 24883 N50: 77874 Median: 4008 Largest: 203060 Contigs >= 500 nt Number: 167 Total length: 5415213 Average: 32426 N50: 77874 Median: 11262 Largest: 203060 Scaffolds >= 100 nt Number: 173 Total length: 5433800 Average: 31409 N50: 135589 Median: 3225 Largest: 376055 Scaffolds >= 500 nt Number: 122 Total length: 5424464 Average: 44462 N50: 135589 Median: 8941 Largest: 376055 Results in discrepancy: grep '>' Contigs.fasta | wc -l 107 "Contigs.fasta" 47454L, 2881955C ~2.9MB (5.4 reported)

[mscook]

Hi Seb,

It was a consequence of 2).

I believe Ray should halt however.

p.s. Out of interest, preliminary results suggest:

Newbler: 454 > Newbler 454+Ill ~= Ray: Ill+454 >> Ray: Ill >>> Velvet: Ill

We have exceptional 454 data with very bad Ill data. I'm speculating, but with below par 454 data we may get something :

Ray: Ill+454 >> Newbler: 454 >> Newbler 454+Ill > Ray: Ill ~= Velvet: Ill

[sebhtml]

Hello,

I checked with v1.7 and with the master branch (head is 21ed820b3dc6ccd72fcd1d89e1dadebaedf9680f )

In both cases, Ray first says

Rank 0 Error: joeasdsa.fastq failed to load properly...

and then, it says Ray: cannot access 'joeasdsa.fastq': No such file or directory

and the computation is stopped.

So, I presume that in your case, the file was available at some point during the computation, but then became unavailable.

with v1.7

---

Step: Network testing Date: Thu Feb 9 11:40:33 2012 Elapsed time: 1 seconds Since beginning: 1 seconds

---

Ray: cannot access 'joeasdsa.fastq': No such file or directory Rank 0 Error: joeasdsa.fastq failed to load properly... Rank 0: File joeasdsa.fastq (Number 0) has 0 sequences Rank 1: File _2.fasta (Number 1) has 10000 sequences Rank 0 wrote tektest21/NumberOfSequences.txt Rank 0 wrote tektest21/SequencePartition.txt

---

Step: File partitioning Date: Thu Feb 9 11:40:33 2012 Elapsed time: 0 seconds Since beginning: 1 seconds

---

Ray: cannot access 'joeasdsa.fastq': No such file or directory Rank 15: sent 31967 messages, received 31968 messages. Rank 14: sent 31872 messages, received 31873 messages. Rank 13: sent 32044 messages, received 32045 messages. Rank 12: sent 31863 messages, received 31864 messages. Rank 11: sent 31864 messages, received 31865 messages. Rank 10: sent 31955 messages, received 31956 messages. Rank 9: sent 32035 messages, received 32036 messages. Rank 8: sent 32191 messages, received 32192 messages. Rank 7: sent 31861 messages, received 31862 messages. Rank 6: sent 32271 messages, received 32272 messages. Rank 5: sent 32036 messages, received 32037 messages. Rank 4: sent 32102 messages, received 32103 messages. Rank 3: sent 31981 messages, received 31982 messages. Rank 2: sent 32159 messages, received 32160 messages. Rank 1: sent 32080 messages, received 32081 messages. Rank 0: sent 31867 messages, received 31851 messages.