preprocessing problem

[orinp]

I am a new user of cdeep3m. I ran through the tutorials and then attempted to run cdeep3m with a stack of my own images. I started a p2.xlarge instance, uploaded my images and ran:

PreprocessTrainingData.m ~/train/images ~/train/labels ~/augtrain

with the following result:

octave: X11 DISPLAY environment variable not set octave: disabling GUI features Starting Training data Preprocessing Training Image Path: /home/ubuntu/train/images Training Label Path: /home/ubuntu/train/labels Output Path: /home/ubuntu/augtrain Loading: /home/ubuntu/train/images Image importer loading ... /home/ubuntu/train/images Reading file: /home/ubuntu/train/images/testimages.000.png ... Reading file: /home/ubuntu/train/images/testimages.208.png error: out of memory or dimension too large for Octave's index type error: called from imageimporter at line 40 column 31 /home/ubuntu/cdeep3m/PreprocessTrainingData.m at line 45 column 11

Each testimage is an 8 bit grayscale EM image of neuropil with a resolution of 4084x2518 pixels. It is ~10MB in size. The stack has 333 images.

I retried with a p3.2xlarge instance which failed at exactly the same point so I doubt it is a lack of instance memory.

Any ideas for moving past this?

Thanks.

Orin Packer Dept. of Biological Structure U. of Washington Seattle

[coleslaw481]

Hi, Training can was setup to work on 100 1k x 1k images. Were you trying to run training (runtraining.sh) on the larger images or did you mean to run prediction (runprediction.sh)? Prediction shouldn't have a problem with 4k x 2k images.

[orinp]

HI,

Thanks for your reply.

I wanted to train on the larger images. I could reduce the xy dimensions some but the cells I am trying to segment are neurons with long dendrites and axons and span well over 100 images in z. 330 images spans this particular cell type but there are other cell types that span even more sections. And reducing the xy extent too much would reduce the number of labels for training by quite a bit. For that reason I really need to train on fairly large stacks.

I am new to working with these models so perhaps my expectations are unrealistic but for cdeep3m to work for me I probably need to train stacks of this size and to predict stacks of even larger size.

I was hoping that maybe there was a simple way to allocate more memory.

-Orin Packer

On Dec 12, 2018, at 10:23 AM, Chris Churas notifications@github.com wrote:

Hi, Training can was setup to work on 100 1k x 1k images. Were you trying to run training (runtraining.sh) on the larger images or did you mean to run prediction (runprediction.sh)? Prediction shouldn't have a problem with 4k x 2k images.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[haberlmatt]

Hi, without knowing what your images look like its hard to recommend how training data should be organized best (If you'd like you can send an example to cdeep3m@gmail.com) but if I understand correctly the neurons you have labeled are large in x/y and z (and therefore will be much larger than the typical size of a receptive field of deep neural networks. The issue here will become GPU memory). I would try if you can downsample the data to about 1/4th of the size in x/y, I suspect that you will still see the axons at that scale.