tensorflow / adanet

Fast and flexible AutoML with learning guarantees.
https://adanet.readthedocs.io
Apache License 2.0
3.47k stars 529 forks source link

tf.Session(config=..) always takes all of the memory even through I set up the gpu_options #127

Closed eustomaqua closed 4 years ago

eustomaqua commented 4 years ago

Problem: gpu_options doesn't work. I used the codes in AdaNet like

GPU_OPTIONS = tf.GPUOptions(allow_growth=True)
CONFIG = tf.ConfigProto(gpu_options=GPU_OPTIONS)
sess = tf.Session(config = CONFIG)

But it still occupied the whole gpu memory when I run the AdaNet. For example: (1) If there is only 4G memory left, it will occupied the remaining 6G memory (2) If there is no other process occupying the GPU memory (i.e., there is 10G memory left), it will occupied the whole 10G memory when I run the AdaNet. So I think it doesn't need 10G to run but it takes that much any way.

I expect it only takes the memory it needs, instead of taking all of them. Therefore, I could make use of the 10G GPU memory better. But now, I have no idea how to fix this issue. Could anybody please give me some suggestions? Thanks a lot.

Other system information:

cweill commented 4 years ago

Fixed by #133