Fail on first iternation

[DcShepherd]

Hi, I have followed the tutorial and have gotten to the refinement stage. In the refinement tab, I fill in the required information, leave the default values, and hit the "Refine button", which changes to a red "Stop" button. The program starts, and I see a spike in my disc and CPU usage. After a while, the log updates to say that it has begun fitting. However, after about five minutes, the "Stop" button reverts to "Refine", and I see my CPU usage go back to baseline. I don't see any error messages. I left it alone for about an hour to see if it updated, but it did not.

01-11 19:11:53, INFO Start Iteration1! /home/melb/clones/IsoNet/bin/isonet.py refine ./subtomo.star --gpuID 0 --result_dir ./HIVtomo/testresults --preprocessing_ncpus 10 --iterations 30 --epochs 10 --learning_rate 0.0004 --noise_level 0.05,0.1,0.15,0.2 --noise_start_iter 10,15,20,25 --noise_mode ramp --drop_out 0.3 --unet_depth 3 --convs_per_depth 3 --kernel 3,3,3 --filter_base 64 01-11 19:18:54, INFO

Isonet starts refining

01-11 19:18:56, INFO Done preperation for the first iteration! 01-11 19:18:56, INFO Start Iteration1! 01-11 19:21:44, INFO Noise Level:0.0 01-11 19:21:55, INFO Done preparing subtomograms! 01-11 19:21:55, INFO Start training! 01-11 19:21:55, INFO Loaded model from disk 01-11 19:21:55, INFO begin fitting

[procyontao]

Hi,

I believe you may have some problems like insufficient GPU memory Please run in command line and with parameter: --log_level debug, and check the output.

[DcShepherd]

Thank you, that was the problem. I thought my GPUs had enough memory to handle the default cube size, but they sadly don't. However, when I ran the program with a cube size of 32, it ran without problems.