search

crowdAI / marLo

Multi Agent Reinforcement Learning using MalmÖ
MIT License
244 stars 46 forks source link

Memory usage increases while running DQN.py #80

Open Zherui-Liu opened 5 years ago

Zherui-Liu commented 5 years ago

The memory usage is constantly increasing. My computer is 16G memory, but I can't complete the step=10**5 training. Is there any way to solve the problem that the memory will increase during the training?

image image image image image

The above is the change of memory usage when I run. When the current memory usage reaches 70%, the agent will show "missed 7 ~ 9 observations(s)".

image

And when I try to use "memory-profiler" to monitor memory changes, it shows that the agent.act_and_train() method in marlo\experiments\train_agent.py continues to increase memory usage as the loop progresses, but never releases it.

I don't know if anyone has tried to solve this problem. THANKS~

spMohanty commented 5 years ago

@AndKram : Any idea ?

Zherui-Liu commented 5 years ago

I've tried to probe every variable in this function, but I was quickly fainted and it was difficult to know exactly which variable was consuming more memory.

So, I am now trying to batch training, similar to training 10000 step per time, then save the model, extract the model, and restart the training. According to my observation, the model or the agent after re-extracting is not effective at the beginning, but it will recover soon to the previous level and continue to learn.

hope this will be ok...