search

bgeetika / Captcha-Decoder

81 stars 48 forks source link

theano out of mem #1

Open yqMac opened 7 years ago

yqMac commented 7 years ago

I had done what I could do,but I really counldn't fix it ; 1、Runing perfect on cpu . 2、Gpu will show MemoryError 3、Win10 , 64 , nvida.

tried ： 1、 upgrade theano 2、set lib.cnmem=0.5

ex: Training
Traceback (most recent call last):
File "captcha_cracker.py", line 319, in
Run(arguments)
File "captcha_cracker.py", line 245, in Run
use_mask_input=args.use_mask_input)
File "captcha_cracker.py", line 56, in Train
l_err = train_fn(*train_inputs)
File "D:\CodeTools\Tools\Anaconda\lib\site-packages\theano\compile\function_module.py", line 898, in call
storage_map=getattr(self.fn, 'storage_map', None))
File "D:\CodeTools\Tools\Anaconda\lib\site-packages\theano\gof\link.py", line 325, in raise_with_op
reraise(exc_type, exc_value, exc_trace)
File "D:\CodeTools\Tools\Anaconda\lib\site-packages\theano\compile\function_module.py", line 884, in call
self.fn() if output_subset is None else\
MemoryError: Error allocating 131788800 bytes of device memory (CNMEM_STATUS_OUT_OF_MEMORY).
Apply node that caused the error: GpuAllocEmpty(Shape_i{0}.0, Shape_i{1}.0, TensorConstant{33}, TensorConstant{78})
Toposort index: 75
Inputs types: [TensorType(int64, scalar), TensorType(int64, scalar), TensorType(int64, scalar), TensorType(int64, scala r)]
Inputs shapes: [(), (), (), ()]
Inputs strides: [(), (), (), ()]
Inputs values: [array(400L, dtype=int64), array(32L, dtype=int64), array(33L, dtype=int64), array(78L, dtype=int64)]
Outputs clients: [[GpuDnnConvGradI{algo='none', inplace=True}(GpuContiguous.0, GpuContiguous.0, GpuAllocEmpty.0, GpuDnn ConvDesc{border_mode='valid', subsample=(1, 1), conv_mode='conv', precision='float32'}.0, Constant{1.0}, Constant{0.0}) ]]

HINT: Re-running with most Theano optimization disabled could give you a back-trace of when this node was created. This can be done with by setting the Theano flag 'optimizer=fast_compile'. If that does not work, Theano optimizations can be disabled with 'optimizer=None'.
HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of this apply node.

navika15 commented 7 years ago

Hey @yqMac ....the individual files in this project are running fine....could you tell me how to run the project to get the output of the captcha decoder? Thanks in advance.