Problem in deploying synapse_btcv_abdominal_ct_segmentation

[chiranjeev1111]

When I deploy for the above examples with this code: python deploy.py -p ./model -e ./ -c CUDA_DEVICE --csv train.csv

I get an cmd like this: /usr/local/lib/python3.6/dist-packages/h5py/init.py:36: FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type. from ._conv import register_converters as _register_converters First Loading from ./model Second Got y_prob as Tensor("pred/Reshape_1:0", shape=(?, 64, 64, 64, 14), dtype=float32) Tensor("pred/Reshape_1:0", shape=(?, 64, 64, 64, 14), dtype=float32) Third running inference on Tensor("Placeholder:0", shape=(?, 64, 64, 64, 1), dtype=float32) with img (1, 214, 175, 175, 1) and op Tensor("pred/Reshape_1:0", shape=(?, 64, 64, 64, 14), dtype=float32) tcmalloc: large alloc 2416222208 bytes == 0xcae10000 @ 0x7fcc283a6107 0x7fcbfcf47385 0x7fcbffb09583 0x7fcbffb25705 0x7fcbffbe3ff3 0x7fcbfad43c2c 0x7fcbfad05fc5 0x7fcbfacf3db5 0x7fcbfa949d81 0x7fcbfa947b47 0x7fcc26ce94a0 0x7fcc2815b7fc 0x7fcc272e2b5f (nil) tcmalloc: large alloc 2416222208 bytes == 0xcae10000 @ 0x7fcc283a6107 0x7fcbfcf47385 0x7fcbffb09583 0x7fcbffb25705 0x7fcbffbe3ff3 0x7fcbfad43c2c 0x7fcbfad05fc5 0x7fcbfacf3db5 0x7fcbfa949d81 0x7fcbfa947b47 0x7fcc26ce94a0 0x7fcc2815b7fc 0x7fcc272e2b5f (nil) tcmalloc: large alloc 2416222208 bytes == 0x86e10000 @ 0x7fcc283a6107 0x7fcbfcf47385 0x7fcbffb09583 0x7fcbffb25705 0x7fcbffbe3ff3 0x7fcbfad43c2c 0x7fcbfad05fc5 0x7fcbfacf3db5 0x7fcbfa949d81 0x7fcbfa947b47 0x7fcc26ce94a0 0x7fcc2815b7fc 0x7fcc272e2b5f (nil) tcmalloc: large alloc 2416222208 bytes == 0x86e10000 @ 0x7fcc283a6107 0x7fcbfcf47385 0x7fcbffb09583 0x7fcbffb25705 0x7fcbffbe3ff3 0x7fcbfad43c2c 0x7fcbfad05fc5 0x7fcbfacf3db5 0x7fcbfa949d81 0x7fcbfa947b47 0x7fcc26ce94a0 0x7fcc2815b7fc 0x7fcc272e2b5f (nil) tcmalloc: large alloc 2416222208 bytes == 0x86e10000 @ 0x7fcc283a6107 0x7fcbfcf47385 0x7fcbffb09583 0x7fcbffb25705 0x7fcbffbe3ff3 0x7fcbfad43c2c 0x7fcbfad05fc5 0x7fcbfacf3db5 0x7fcbfa949d81 0x7fcbfa947b47 0x7fcc26ce94a0 0x7fcc2815b7fc 0x7fcc272e2b5f (nil) tcmalloc: large alloc 2416222208 bytes == 0x86e10000 @ 0x7fcc283a6107 0x7fcbfcf47385 0x7fcbffb09583 0x7fcbffb25705 0x7fcbffbe3ff3 0x7fcbfad43c2c 0x7fcbfad05fc5 0x7fcbfacf3db5 0x7fcbfa949d81 0x7fcbfa947b47 0x7fcc26ce94a0 0x7fcc2815b7fc 0x7fcc272e2b5f (nil) tcmalloc: large alloc 2416025600 bytes == 0x76e10000 @ 0x7fcc283a6107 0x7fcbfcf47385 0x7fcbffb09583 0x7fcbffb25705 0x7fcbffb467a2 0x7fcbfad43c2c 0x7fcbfad05fc5 0x7fcbfacf3db5 0x7fcbfa949d81 0x7fcbfa947b47 0x7fcc26ce94a0 0x7fcc2815b7fc 0x7fcc272e2b5f (nil) tcmalloc: large alloc 19327426560 bytes == 0x19fe5a000 @ 0x7fcc283a6107 0x7fcbfcf47385 0x7fcbffb09583 0x7fcbffb25705 0x7fcbffb467a2 0x7fcbfad43c2c 0x7fcbfad05fc5 0x7fcbfacf3db5 0x7fcbfa949d81 0x7fcbfa947b47 0x7fcc26ce94a0 0x7fcc2815b7fc 0x7fcc272e2b5f (nil) Killed

Why am I getting Killed at the end? Help required

[mrajchl]

Hi there, firstly I do see a warning about the hdf5 python wrapper. Did you modify the source? If yes it would be good to see what you were modifying. Secondly, it seems as the tcmalloc messages are related to allocating large chunks of memory and you might exhaust the host memory (RAM) during running.

Did you alter any parameters in the scripts? Could you check your memory usage on the machine?

[chiranjeev1111]

Actually I am running this code on google colaboratory which gives me 12GB gpu memory and I haven't changed any modifications to the code

[pawni]

Interesting. Normally 'Killed' is because of RAM and not GPU memory.

Do you have a link to the colab? Or could you test the code on another machine or maybe even just on a CPU version?

[chiranjeev1111]

https://colab.research.google.com/drive/1ahxJl4poRYxlSTxgmroJ7yLp0_Y5NPch This is the link to my colab.. check if you think something is wrong in there.. I am a beginner and its difficult for me to understand

[pawni]

Generally it should work but I believe that Colab notebooks only have < 16 GB memory available which I think might be too little to run that code.

Do you have another machine you could try?

[chiranjeev1111]

I dont have another machine... my machine configs are too less than what colab is providing me with