(LowClouds) train.npy loading gets the process killed

krisrs1128 / clouds_dist

Simulation of low-clouds, from weather measures.

4 stars 1 forks source link

(LowClouds) train.npy loading gets the process killed #161

Closed jassiene-mila closed 4 years ago

jassiene-mila commented 4 years ago

This line gets the process killed when I try to load the low clouds dataset (see the screenshot below)

(The second try is after commenting the problematic line)

jassiene-mila commented 4 years ago

I have noticed that it fails only when I run it after beeing allocated a GPU : if I run it on a login node there is no issue. When I checked the free memory (using "free -m") in the two configurations (login node and GPU), I had more free memory on the GPU config so I think it may not be a RAM issue

jassiene-mila commented 4 years ago

It was indeed a RAM issue : allocating 32GB instead of 16GB fixed the problem (the amount of memory outputted by "free -m" is for the entire node, that is shared among the node users)