search

torch / torch7

http://torch.ch
Other
9k stars 2.38k forks source link

Buy New RAM! #942

Closed richardrasu closed 7 years ago

richardrasu commented 7 years ago

Hi,

I'm fiddling about with various neural-style repos which depend on torch. Torch has, up until now, been problem free, however on a well-endowed machine I ran a transfer for a very large end image size, and after a while received this message:

/home/ric8ard/torch/install/share/lua/5.1/nn/THNN.lua:109: $ Torch: not enough memory: you tried to reallocate 36GB. Buy new RAM! at /home/ric8ard/torch/pkg/torch/lib/TH/THGeneral.c:251

I wouldn't mind so much, but I've got approximately 224GB RAM free (it's a server with 256GB in it). I'm not seeing any obvious signs of memory exhaustion while it's running. I'm on Ubuntu 16.04.2 LTS. Any quick fixes to this, or am I just expecting too much out of my output resolution (9000 px)?

Thanks for any assistance you can offer.

Richard

rave78 commented 7 years ago

Have you already try https://github.com/fmassa/optimize-net? "OptNet - Reducing memory usage in torch neural nets"

richardrasu commented 7 years ago

@rave78 I have not yet, but I shall. Thanks for the pointer. That looks useful.

richardrasu commented 7 years ago

OK, so I was being thick and labouring under the illusion that I had unlimited everything on this machine, but apparently I was wrong. Apologies to all who wasted their time reading this.