ERROR (theano.gof.cmodule): [Errno 12] Cannot allocate memory

[lisongze]

Dear miriambellver, thanks for your share. I met a problem when ran your code: Problem occurred during compilation with the command line below: /usr/bin/g++ -shared -g -O3 -fno-math-errno -Wno-unused-label -Wno-unused-variable -Wno-write-strings -march=core-avx-i -mcx16 -msahf -mno-movbe -maes -mpclmul -mpopcnt -mno-abm -mno-lwp -mno-fma -mno-fma4 -mno-xop -mno-bmi -mno-bmi2 -mno-tbm -mavx -mno-avx2 -msse4.2 -msse4.1 -mno-lzcnt -mno-rtm -mno-hle -mrdrnd -mf16c -mfsgsbase -mno-rdseed -mno-prfchw -mno-adx -mfxsr -mxsave -mxsaveopt --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=3072 -mtune=core-avx-i -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -m64 -fPIC -I/usr/local/lib/python2.7/dist-packages/numpy/core/include -I/usr/include/python2.7 -I/usr/local/lib/python2.7/dist-packages/theano/gof -fvisibility=hidden -o /home/leemo/.theano/compiledir_Linux-3.16--generic-x86_64-with-Ubuntu-14.04-trusty-x86_64-2.7.6-64/tmprBtt4Q/af8288e23d4f9216c7a523f6c00e6cf5.so /home/leemo/.theano/compiledir_Linux-3.16--generic-x86_64-with-Ubuntu-14.04-trusty-x86_64-2.7.6-64/tmprBtt4Q/mod.cpp -L/usr/lib -lpython2.7 ERROR (theano.gof.cmodule): [Errno 12] Cannot allocate memory Traceback (most recent call last): File "image_zooms_testing.py", line 98, in <module> state = get_state(region_image, history_vector, model_vgg) File "/home/leemo/git-repo/detection-2016-nipsws/scripts/reinforcement.py", line 46, in get_state descriptor_image = get_conv_image_descriptor_for_image(image, model_vgg) File "/home/leemo/git-repo/detection-2016-nipsws/scripts/features.py", line 198, in get_conv_image_descriptor_for_image _convout1_f = K.function(inputs, [model.layers[31].output]) File "/usr/local/lib/python2.7/dist-packages/keras/backend/theano_backend.py", line 802, in function return Function(inputs, outputs, updates=updates, **kwargs) File "/usr/local/lib/python2.7/dist-packages/keras/backend/theano_backend.py", line 788, in __init__ **kwargs) File "/usr/local/lib/python2.7/dist-packages/theano/compile/function.py", line 320, in function output_keys=output_keys) File "/usr/local/lib/python2.7/dist-packages/theano/compile/pfunc.py", line 479, in pfunc output_keys=output_keys) File "/usr/local/lib/python2.7/dist-packages/theano/compile/function_module.py", line 1777, in orig_function defaults) File "/usr/local/lib/python2.7/dist-packages/theano/compile/function_module.py", line 1641, in create input_storage=input_storage_lists, storage_map=storage_map) File "/usr/local/lib/python2.7/dist-packages/theano/gof/link.py", line 690, in make_thunk storage_map=storage_map)[:3] File "/usr/local/lib/python2.7/dist-packages/theano/gof/vm.py", line 1003, in make_all no_recycling)) File "/usr/local/lib/python2.7/dist-packages/theano/gof/op.py", line 970, in make_thunk no_recycling) File "/usr/local/lib/python2.7/dist-packages/theano/gof/op.py", line 879, in make_c_thunk output_storage=node_output_storage) File "/usr/local/lib/python2.7/dist-packages/theano/gof/cc.py", line 1200, in make_thunk keep_lock=keep_lock) File "/usr/local/lib/python2.7/dist-packages/theano/gof/cc.py", line 1143, in __compile__ keep_lock=keep_lock) File "/usr/local/lib/python2.7/dist-packages/theano/gof/cc.py", line 1595, in cthunk_factory key=key, lnk=self, keep_lock=keep_lock) File "/usr/local/lib/python2.7/dist-packages/theano/gof/cmodule.py", line 1142, in module_from_key module = lnk.compile_cmodule(location) File "/usr/local/lib/python2.7/dist-packages/theano/gof/cc.py", line 1506, in compile_cmodule preargs=preargs) File "/usr/local/lib/python2.7/dist-packages/theano/gof/cmodule.py", line 2182, in compile_str p_out = output_subprocess_Popen(cmd) File "/usr/local/lib/python2.7/dist-packages/theano/misc/windows.py", line 75, in output_subprocess_Popen p = subprocess_Popen(command, **params) File "/usr/local/lib/python2.7/dist-packages/theano/misc/windows.py", line 36, in subprocess_Popen proc = subprocess.Popen(command, startupinfo=startupinfo, **params) File "/usr/lib/python2.7/subprocess.py", line 710, in __init__ errread, errwrite) File "/usr/lib/python2.7/subprocess.py", line 1223, in _execute_child self.pid = os.fork() OSError: [Errno 12] Cannot allocate memory I used a GT750M GPU which has 4GB memory to run this code, but seems the GPU memory is still not enough, so can you tell me how many resource have you used when you ran your code, have you ever met this problem before? BTW, I used theano as bakcend. Using Theano backend. Using gpu device 0: GeForce GT 750M (CNMeM is enabled with initial size: 95.0% of memory, cuDNN 5005)

[miriambellver]

Hi!

No, I haven't met this problem before, but what I use is a K80. The problem may be that VGG-16 does not fit in the memory of your GPU.

Míriam

[lisongze]

Well, thx. Maybe I should try another machine which has larger GPU memory.