search

abria / TeraStitcher

A tool for fast automatic 3D-stitching of teravoxel-sized microscopy images
http://abria.github.io/TeraStitcher/
Other
78 stars 32 forks source link

Teraconverter 1.10.2 crash on saving resolution 1 #17

Closed matsojr22 closed 7 years ago

matsojr22 commented 7 years ago

I am merging a very large data set using teraconverter in the commandline. I have aligned the data successfully using the GUI, but since I have 2 channel tiffs I am using the CLI to merge.

When I navigate to my image source folder and run the command:

teraconverter --sfmt="TIFF (unstitched, 3D)" -s="xml_merging.xml" --dfmt="TIFF (series, 2D)" -d="E:\testout"

The software makes it through the initilization stage, but crashes in windows as soon as the "Saving to disc resolution 1" phase begins.

tera crash

There are no error messages and I am unaware of a log file. I have tried this on my local image processing workstation, as well as an extremely powerful workstation in our biophotonics core facility. I do not think this is an issue with the computer or a lack of resources. I think I must be triggering some bug.

I can also include my xml files and mdata. xml and bin.zip

Thank you for your help, Matt Jacobs

iannellog commented 7 years ago

Dear Matt,

2017-06-12 23:14 GMT+02:00 Matt Jacobs notifications@github.com:

I am merging a very large data set using teraconverter in the commandline. I have aligned the data successfully using the GUI, but since I have 2 channel tiffs I am using the CLI to merge.

for CLI you mean "command line"? If you use the last GUI version (1.10.2) you do not need any more to use the command line version of teraconverter: it has been integrated with the first five steps of terastitcher. Look at the attached draft user guide for more information.

When I navigate to my image source folder and run the command:

teraconverter --sfmt="TIFF (unstitched, 3D)" -s="xml_merging.xml" --dfmt="TIFF (series, 2D)" -d="E:\testout"

The software makes it through the initilization stage, but crashes in windows as soon as the "Saving to disc resolution 1" phase begins.

[image: tera crash] https://user-images.githubusercontent.com/7969848/27055275-b81c85ae-4f78-11e7-9107-418544ed3ad4.JPG

There are no error messages and I am unaware of a log file. I have tried this on my local image processing workstation, as well as an extremely powerful workstation in our biophotonics core facility. I do not think this is an issue with the computer or a lack of resources. I think I must be triggering some bug.

I can also include my xml files and mdata. xml and bin.zip https://github.com/abria/TeraStitcher/files/1069550/xml.and.bin.zip

I had a look to the xml_merging.xml file and it looks ok. The only thing is that you mentioned 2 channels, but probably images are in RGB tiffs with the third channel empty. Is it right?

One information I need is how large are tiles to be merged in X-Y (in pixels) and how much memory you have. I need also a confirmation that you ask the generation of the highest resolution only.

Best.

-- Giulio

Giulio Iannello Preside della Facolta' Dipartimentale di Ingegneria Universita' Campus Bio-Medico di Roma v. Alvaro del Portillo, 21 00128 Roma, Italy

Tel: +39-06-22541-9602 E-mail: g.iannello@unicampus.it Fax: +39-06-22541-9609 URL: https://scholar.google.it/citations?user=L-UJxIgAAAAJ

matsojr22 commented 7 years ago

One information I need is how large are tiles to be merged in X-Y (in pixels) and how much memory you have. I need also a confirmation that you ask the generation of the highest resolution only.

The tiles are 1300x1300px with a 10% overlap. We have had very good results with your software so far, this is the first time we have experienced a crash.

The only thing is that you mentioned 2 channels, but probably images are in RGB tiffs with the third channel empty. Is it right?

Correct, the images are RGB tiffs with the B channel empty.

I will try with the GUI to see if I get the same error and get back to you today.

Thank you!

matsojr22 commented 7 years ago

Oops, I forgot to tell you how much memory I have. This computer has 128Gb of RAM.

I have now tried the 1.10.2 GUI to do the merging steps as well.

terastitcher 1 10 2 gui error

I can only imagine that there must be some problem with the size of my files, this is the largest image set I have ever tried to reconstruct.

I really appreciate all of your help!

iannellog commented 7 years ago

I forgot the attachment. Sorry.

-- Giulio

2017-06-12 23:14 GMT+02:00 Matt Jacobs notifications@github.com:

I am merging a very large data set using teraconverter in the commandline. I have aligned the data successfully using the GUI, but since I have 2 channel tiffs I am using the CLI to merge.

When I navigate to my image source folder and run the command:

teraconverter --sfmt="TIFF (unstitched, 3D)" -s="xml_merging.xml" --dfmt="TIFF (series, 2D)" -d="E:\testout"

The software makes it through the initilization stage, but crashes in windows as soon as the "Saving to disc resolution 1" phase begins.

[image: tera crash] https://user-images.githubusercontent.com/7969848/27055275-b81c85ae-4f78-11e7-9107-418544ed3ad4.JPG

There are no error messages and I am unaware of a log file. I have tried this on my local image processing workstation, as well as an extremely powerful workstation in our biophotonics core facility. I do not think this is an issue with the computer or a lack of resources. I think I must be triggering some bug.

I can also include my xml files and mdata. xml and bin.zip https://github.com/abria/TeraStitcher/files/1069550/xml.and.bin.zip

Thank you for your help, Matt Jacobs

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/abria/TeraStitcher/issues/17, or mute the thread https://github.com/notifications/unsubscribe-auth/AIdv1pFl2x-ffNtVhJ_SoTCtDOwenwFmks5sDaoegaJpZM4N3sfq .

--

Giulio Iannello Preside della Facolta' Dipartimentale di Ingegneria Universita' Campus Bio-Medico di Roma v. Alvaro del Portillo, 21 00128 Roma, Italy

Tel: +39-06-22541-9602 E-mail: g.iannello@unicampus.it Fax: +39-06-22541-9609 URL: https://scholar.google.it/citations?user=L-UJxIgAAAAJ

iannellog commented 7 years ago

Ok, thanks. It is likely an allocation problem. Actually, at Allen have converted images images of this size (even bigger, even though I am not sure they were multi-channel). I have to check the code to figure out what could have been happened. I hope to have time tomorrow. In case I introduce some additional check in the code, can you rebuild the executables with CMake (the command line version) or you can use only the precompiled version?

-- Giulio

2017-06-13 21:30 GMT+02:00 Matt Jacobs notifications@github.com:

Oops, I forgot to tell you how much memory I have. This computer has 128Gb of RAM.

I have now tried the 1.10.2 GUI to do the merging steps as well.

-

My first attempt used the default merging settings with (series 2D), and I got a gui error saying "Bad Allocation"

My second attempt also used (series 2D), however I used the advanced settings to only select channels 1 and 2. I turned off the empty third channel. This too ended in a "Bad Allocation" error as soon as the initialization phase finished.

[image: terastitcher 1 10 2 gui error] https://user-images.githubusercontent.com/7969848/27100695-c9f2ccf0-5033-11e7-92bf-17a5891fa8e4.png

I can only imagine that there must be some problem with the size of my files, this is the largest image set I have ever tried to reconstruct.

I really appreciate all of your help!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/abria/TeraStitcher/issues/17#issuecomment-308223420, or mute the thread https://github.com/notifications/unsubscribe-auth/AIdv1gpLLu1i8pMO2wJlxtJDOPSKvNMjks5sDuNTgaJpZM4N3sfq .

--

Giulio Iannello Preside della Facolta' Dipartimentale di Ingegneria Universita' Campus Bio-Medico di Roma v. Alvaro del Portillo, 21 00128 Roma, Italy

Tel: +39-06-22541-9602 E-mail: g.iannello@unicampus.it Fax: +39-06-22541-9609 URL: https://scholar.google.it/citations?user=L-UJxIgAAAAJ

matsojr22 commented 7 years ago

I am currently using the precompiled version. I am not familiar with CMake on the command line, however if need be I can learn to use it.

abria commented 7 years ago

I will rebuild the binaries on the fly and put the links here, so that Matt can test if it works.

abria commented 7 years ago

Matt,

please try this new version 10.3 (not yet released) and let us know.

Thanks.

matsojr22 commented 7 years ago

Fantastic, I'll give it a try in just a few minutes. Thank you for the help!

matsojr22 commented 7 years ago

The new version seems to be running smoothly as usual. I am very grateful for your help! Please let me know if there is any further information that I can provide, else I would say this issue is safe to close.

iannellog commented 7 years ago

I thank you for your testing! I had not tested so far the generation of large 2D slices and some indices were coded as signed int (max value 2^31-1). Do not hesitate to contact us again if observe other anomalies. Best.

-- Giulio

2017-06-15 22:01 GMT+02:00 Matt Jacobs notifications@github.com:

The new version seems to be running smoothly as usual. I am very grateful for your help! Please let me know if there is any further information that I can provide, else I would say this issue is safe to close.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/abria/TeraStitcher/issues/17#issuecomment-308850433, or mute the thread https://github.com/notifications/unsubscribe-auth/AIdv1jPjyIivQcZI1x2cXZqLYTpuEPmdks5sEY2egaJpZM4N3sfq .

--

Giulio Iannello Preside della Facolta' Dipartimentale di Ingegneria Universita' Campus Bio-Medico di Roma v. Alvaro del Portillo, 21 00128 Roma, Italy

Tel: +39-06-22541-9602 E-mail: g.iannello@unicampus.it Fax: +39-06-22541-9609 URL: https://scholar.google.it/citations?user=L-UJxIgAAAAJ