jayleicn / ClipBERT

[CVPR 2021 Best Student Paper Honorable Mention, Oral] Official PyTorch code for ClipBERT, an efficient framework for end-to-end learning on image-text and video-text tasks.
https://arxiv.org/abs/2102.06183
MIT License
697 stars 85 forks source link

When run finetuning task MSRVTT retrieval,an error occurred: "Failed Resource Temporarily Unavailable". #14

Closed MrZihan closed 3 years ago

MrZihan commented 3 years ago

We found the reason is that the dataloader constantly create worker threads, however ,the threads can't exit normally.So when the number of threads exceed the upper limit,run_video_retrieval.py will exit unexpectedly. We use your docker image to run the program,have you ever had such problem before? Thanks!

jayleicn commented 3 years ago

Hi @MrZihan,

We did not meet this issue before. It would be great if you can post your solution, if any, to this issue so that other people can benefit from it.

Best, Jie

Curry-AI commented 3 years ago

I met the same problem

HellwayXue commented 3 years ago

Hey guys, any solutions?

wykang commented 3 years ago

In my case, I solved this problem by adding "video_container.close()" to just before the return line in this function.

Since the "av.open(in_mem_bytes_io, metadata_errors="ignore")" line opens the file streams without any close() code or proper garbage collection, this will cause the thread exploding problem.

HellwayXue commented 3 years ago

In my case, I solved this problem by adding "video_container.close()" to just before the return line in this function.

Since the "av.open(in_mem_bytes_io, metadata_errors="ignore")" line opens the file streams without any close() code or proper garbage collection, this will cause the thread exploding problem.

Yeah, I've solved it exactly in this way yesterday. Thank you anyway.

Siaaaaaa1 commented 3 weeks ago

I meet the same problem,but I can not run this code in this way....

In my case, I solved this problem by adding "video_container.close()" to just before the return line in this function.

Since the "av.open(in_mem_bytes_io, metadata_errors="ignore")" line opens the file streams without any close() code or proper garbage collection, this will cause the thread exploding problem.