Tide failure in SearchGUI 1.27.4

[GoogleCodeExporter]

> What steps will reproduce the problem?
Running a search that includes TIDE.  

> What is the expected output? What do you see instead?
I expect the search to complete correctly.  Instead I see 
"
Error: Source 
'C:\Users\ealab\Desktop\SearchGUI-1.27.4\resources\Tide\windows\crux-output\Samp
le05.tide-search.target.txt' does not exist
An error occurred while running SearchGUI.  Please contact the developers.
The search or post-processing did not finish properly!

Search Canceled!
"

> What version of the product are you using? On what operating system?
SearchGUI 1.27.4 in windows (windows 7 64bit enterprise)

> Please provide any additional information below.
Comet searches are not effected.  Running Comet alone, or with TIDE, does not 
impact the Comet results.  However running TIDE alone or with Comet will result 
in the TIDE search failing.  

> If the reported issue resulted in the tool crashing, please
> also upload the file called SearchGUI.log (found in the
> SearchGUI-X.Y.Z\resources folder).

Original issue reported on code.google.com by lparson...@gmail.com on 3 Jun 2015 at 2:23

[GoogleCodeExporter]

Thanks for telling us about this. I'm not able to reproduce the error on my 
side though. Could you send me the complete content in the SearchGUI Progress 
Dialog when running the Tide search? It would also be helpful if you could send 
me the SearchGUI log file (Help > Bug Report). If the files don't contain too 
much text you can simply paste the content into the issue report. Or you can 
send them directly to me at harald.barsnes at gmail.com.

Original comment by harald.b...@gmail.com on 4 Jun 2015 at 9:13

[GoogleCodeExporter]

Harald

Thank you for the reply.  I have attached the bug report to this email.  I
am rerunning the same process so I can send you the progress dialog when it
stops.

Original comment by lparson...@gmail.com on 4 Jun 2015 at 1:46

[GoogleCodeExporter]

Harald

It failed very quickly.  The following is what is in the progress dialog:

Thu Jun 04 08:45:13 CDT 2015        Indexing
MOUSE_concatenated_target_decoy.fasta for Tide.

INFO: Beginning tide-index.
WARNING: The output directory 'crux-output' already exists.
Existing files will be overwritten.
WARNING: The file 'crux-output/tide-index.log.txt' already exists and will
be overwritten.
INFO: CPU: Tower
INFO: Thu 06/04/2015

WARNING: The file 'crux-output/tide-index.params.txt' already exists and
will be overwritten.
INFO: Running tide-index...
WARNING: The output directory 'fasta-index' already exists.
Existing files will be overwritten.
INFO: Reading
C:\Users\ealab\Desktop\Proteomics\MOUSE_concatenated_target_decoy.fasta and
computing unmodified peptides...
FATAL: An exception occurred: bad allocation

Thu Jun 04 08:46:14 CDT 2015        Tide Indexing Process Finished.

Thu Jun 04 08:46:14 CDT 2015        Converting spectrum file Sample05.mgf
for Comet/Tide.
Thu Jun 04 08:46:23 CDT 2015        Processing Sample05.mgf with Tide.

INFO: Beginning tide-search.
WARNING: The output directory 'crux-output' already exists.
Existing files will be overwritten.
WARNING: The file 'crux-output/Sample05.tide-search.log.txt' already exists
and will be overwritten.
INFO: CPU: Tower
INFO: Thu 06/04/2015

WARNING: The file 'crux-output/Sample05.tide-search.params.txt' already
exists and will be overwritten.
INFO: Running tide-search...
INFO: Reading index fasta-index
FATAL: Error reading index (fasta-index/auxlocs)

Thu Jun 04 08:46:24 CDT 2015        Tide Searching Process Finished.

Thu Jun 04 08:46:24 CDT 2015        Error: Source
'C:\Users\ealab\Desktop\SearchGUI-1.27.4\resources\Tide\windows\crux-output\Samp
le05.tide-search.target.txt'
does not exist
Thu Jun 04 08:46:24 CDT 2015        An error occurred while running
SearchGUI. Please contact the developers.
Thu Jun 04 08:46:24 CDT 2015        The search or post-processing did not
finish properly!

Thu Jun 04 08:46:24 CDT 2015        Search Canceled!

Original comment by lparson...@gmail.com on 4 Jun 2015 at 1:48

[GoogleCodeExporter]

Thanks for the log information. The problem seems to be that you are running 
out of space on your disk. Hence the "FATAL: An exception occurred: bad 
allocation" message.

The reason is that Tide generates a map of all possible peptide sequences 
before starting the search. This is why the Tide search itself is so quick. 
However, depending on the amount of variable post-translational modifications 
added, the list of peptides can become quite big (several GB), and one can then 
end up running out of disk space. This results in the peptide index not being 
properly created and the Tide search itself will then fail.

We could attempt to pick this up and show a better warning in SearchGUI, but 
this would not really solve the problem. The only thing you can do is add less 
PTMs, or free more disk space.

However, unless you specifically need to use Tide, you can consider using Comet 
instead (and/or other search engines of course). Comet and Tide are both based 
on the SEQUEST search engine code and should results in similar results anyway, 
so you should not get significantly different results.

Original comment by harald.b...@gmail.com on 4 Jun 2015 at 2:01

• Changed state: Fixed

[GoogleCodeExporter]

Thank you for the quick reply.  I knew that Comet and Tide were related 
algorithms, but I did not know that Tide required so much more disk 
space for the same set of search parameters!  The disk that this search 
is running on has ~200GB available, however I am asking for a fair 
number of variable modifications and this seems to be insufficient.

Original comment by lparson...@gmail.com on 4 Jun 2015 at 4:34

[GoogleCodeExporter]

Surprising to hear that you have 200GB available. Maybe you're not running out 
of memory after all. And I just noticed that from the time the indexing starts 
to the time it fails is just one minute. Which should not be enough to fill 
anywhere close to 200GB.

Perhaps the problem is rather that you are not able to delete the Tide temp 
folders. Tide creates two temp folders called "crux-output" and "fasta-index" 
under "..\SearchGUI-X.Y.Z\resources\Tide\windows". Due to the large size the 
folders are deleted as a default when the search is done. (You can however 
change this in the Tide advanced settings available by clicking the settings 
icon at the end of the Tide line in SearchGUI.)

But unless you changed the default settings the folders should have been 
deleted, and unless you're not still accessing the files in these folders you 
should be able to overwrite them.

Try deleting the two folders manually and repeat the search. If it then works, 
try repeating the search yet again. You should then not see any warnings about 
folders or files already existing when creating the index. If you do, there is 
a problem with deleting the Tide temp folders.

Original comment by harald.b...@gmail.com on 4 Jun 2015 at 10:06

• Changed state: Started

[GoogleCodeExporter]

Forgot one thing. You cannot include attachment with when replying to these 
e-mails. So I never got your SearchGUI log file. Please send it to me at 
harald.barsnes at gmail.com. Maybe it can shed some more light on problem.

Original comment by harald.b...@gmail.com on 4 Jun 2015 at 10:15

[GoogleCodeExporter]

Harald

Thank you again for following up on this.

I have been trying a few other permutations on this problem to try to 
see if I can shed light on it from my end.  It appears that Tide, Comet, 
and OMSSA all show this behavior on my systems and will die with "out of 
space" errors even with 100-200GB of storage available.  I do not have 
this problem with X!Tandem or Myrimatch.

I have deleted the folders manually and still seem to run in to this 
problem, I have even changed where the files are written to (including 
the advanced settings regarding scratch space for the search database) 
and that made no difference.

I will send the previous searchgui log to you directly, and the newest 
one will be sent once it crashes as well.

Original comment by lparson...@gmail.com on 9 Jun 2015 at 3:59

[GoogleCodeExporter]

I found that my problem is residing in the various configuration files 
for searchgui.  In particular, the "Resource Settings" - or "Resource 
Folders" - was writing to a location on my file system that had less 
available storage space than other mounted volumes on my same system.  
It appears that the default for most of these folders - particularly 
comet temp - is to write to the same file system where the searchgui 
executable jar file resides.  I changed this to a different location and 
things work much better now.

This information might be particularly helpful for anyone who has 
multiple hard drive or file systems mounted on the system they are 
running searchgui on.

Original comment by lparson...@gmail.com on 10 Jun 2015 at 2:11

[GoogleCodeExporter]

Great! I'll set the issue to fixed then.

Note that not all temp folders can be changed though. Most notably the temp 
folders for some the search engines. For example for Tide. So if you want to 
run Tide on your setup you may have to move the SearchGUI installation (or just 
the Tide installation) to the drive where you have more space available.

Original comment by harald.b...@gmail.com on 11 Jun 2015 at 2:22

• Changed state: Fixed