search

open-mmlab / denseflow

Extracting optical flow and frames
https://open-mmlab.github.io/
MIT License
307 stars 61 forks source link

Keep getting killed #18

Closed makecent closed 4 years ago

makecent commented 4 years ago

I use denseflow to computer tvl1 with images, i.e. enable "input-frames" argument. The process always gets killed by my system: denseflow

No flow images generated after a few hours. Is there any way to limit the burden from denseflow so it won't be killed by the system?

By the way, according to the info from "nvidia-smi", there are lots of free space in GPU. Actually, it seems that only one of two GPUs has been used for denseflow, can I set the gpu number for denseflow?

innerlee commented 4 years ago

how about host memory? how about reduce the batch size to a smaller number, like 32? https://github.com/open-mmlab/denseflow/blob/032c2fd5548595a2c25082aa257777a8a85517ac/include/dense_flow.h#L95

makecent commented 4 years ago

how about host memory? how about reduce the batch size to a smaller number, like 32?

https://github.com/open-mmlab/denseflow/blob/032c2fd5548595a2c25082aa257777a8a85517ac/include/dense_flow.h#L95

I am not sure what does host memory means. Anyway, I solved this problem by splitting images folders into several small groups. Then do denseflow on each groups which only contains images of about 10 videos (originally there are images of 60 videos in destination directory).

Thanks for your kind attention and suggestion!

innerlee commented 4 years ago

the usual memory for cpu, i.e., not the gpu memory

makecent commented 4 years ago

Cpu memory usage is 100%, and after cpu memory get full, the swap memory increase. Finally after swap memory get full, the computer will start to kill denseflow.

I manually changed the value of batch_maxsize to 128 in file dense_flow.h. While in the print log information during executing denseflow still show batch_size=512. Of course, the getting killed problem still exists.

I downloaded the denseflow with setup. So I actually revise the file: app/src/denseflow/include/dense_flow.h. Not sure if this is the correct way. And I use denseflow with some py file in mmaction2 repo

innerlee commented 4 years ago

yeah that is the issue. change that line would help

innerlee commented 4 years ago

you can mimic the setup script https://github.com/innerlee/setup/blob/master/zzdenseflow.sh#L30-L32 , and remember to set the ENV variable https://github.com/innerlee/setup/blob/master/zzdenseflow.sh#L5

innerlee commented 4 years ago

the binary is install in ~/app/bin by default

makecent commented 4 years ago

you can mimic the setup script https://github.com/innerlee/setup/blob/master/zzdenseflow.sh#L30-L32 , and remember to set the ENV variable https://github.com/innerlee/setup/blob/master/zzdenseflow.sh#L5

Do you mean after I change the dense_flow.h file, I need to cmake denseflow again to make the change works?

innerlee commented 4 years ago

make and make install is ok

it depends on where you changed the code. if it was the ~/app/src/denseflow, then make is enough. for a freshly cloned repo, you need start from the cmake

makecent commented 4 years ago

make and make install is ok

it depends on where you changed the code. if it was the ~/app/src/denseflow, then make is enough. for a freshly cloned repo, you need start from the cmake

Cool! That works! Now cpu memory usage gets to a maximum of 40%. Thank you very much.