search

tusen-ai / simpledet

A Simple and Versatile Framework for Object Detection and Instance Recognition
Apache License 2.0
3.09k stars 488 forks source link

resources of SWP don't be released #284

Open zerojuzi opened 4 years ago

zerojuzi commented 4 years ago

I notice resources of SWP don't be released after simpledet training, especially on large data sets training, resources of SWP are full, I must be used swapoff -a and swapon -a to solve the problem. I want to which is bug of mxnet or simpledet? and may you help me solve the problem, thank you very much.

RogerChern commented 4 years ago

I think it is neither the problem of MXNet nor SimpleDet. The swap is managed by the kernel and should not be used during normal training. If you find yourself running low of free memory, you can reduce the number of loader_worker by adding loader_workder = 1 under the General config class.

zerojuzi commented 4 years ago

I used 'free -g' notice buff/cache progressively increase during training when I adding loader_workder = 1 , and free progressively reduce，finally it become 0, so SWP resources begin to be used. I want to know what information is kept in the cache, and can I clear this information. thank you very much. @RogerChern

RogerChern commented 4 years ago

You can free it up freely. Just batches of data.

On Thu, Jan 2, 2020 at 2:28 PM zhangxue123 notifications@github.com wrote:

I used 'free -g' notice buff/cache progressively increase during training when I adding loader_workder = 1 , and free progressively reduce，finally it become 0, so SWP resources begin to be used. I want to know what information is kept in the cache, and can I clear this information. thank you very much. @RogerChern https://github.com/RogerChern

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/TuSimple/simpledet/issues/284?email_source=notifications&email_token=ABGODH4GJKYK7XV3ZBBR623Q3WCRDA5CNFSM4J7IOR72YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEH5XJ4A#issuecomment-570127600, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABGODH2NTYUQALFARZ3CGS3Q3WCRDANCNFSM4J7IOR7Q .