process_video.py temporary folders permission denied

[aweaver1fandm]

Based on the code in your comments it seems as if this is on your TODO list and I also know you can use the --frame_folder and --frame_rendering_folder arguments to override the default behavior of writing temporary files to /tmp/process_camera_trap_video.

That said we ran into file permission errors on Linux when we use the default location when more than one user is trying to use process_video.py. The first user to run the code causes the /tmp/process_camera_trap_video directory to be created with them as the owner and the group as their primary group as the group. When the second user tried to run (again without overriding the location) they got permission denied errors. I am partially opening the issue in case other people run into this problem

[agentmorris]

Good catch. As you highlight, there's an easy workaround, which is to specify a non-default temporary folder, but still, that's an annoying error that shouldn't happen. I added the laziest possible fix, namely after creating the temporary directory, I set it to be world-writable. I confirmed that the permissions get set correctly; I have not actually tried running this as different users. Can you confirm it solves your problem? Closing pre-emptively because I'm 97% sure it will fix it.

Also I forgot how much fun it is to render boxes on videos; here's a raccoon from my yard:

https://lila.science/public/DSCF0008.AVI.detections.mp4

[aweaver1fandm]

The fix didn't work. Where you placed the call for chmod is after the tempdir string is set but before a call a few lines later for os.makedirs, after the frame_output_folder gets handled. Essentially the tempdir isn't created yet where you try to chmod.

When I moved your code after the os.makedirs then it worked. Also, we have a researcher doing things with rats in boxes so we're doing this (see attached). The confidence values are lowish but good enough. It's mostly because the rats aren't in nature.

https://github.com/agentmorris/MegaDetector/assets/55799947/04755031-4926-4af6-bf5c-4877ba773c3e

[agentmorris]

Re: permissions being set in the wrong place...

Oops! My bad. Try again.

Re: rats...

Very cool! As you suggest, right on the edge of what I would expect MD to be able to handle, but glad it's working. This is the kind of case where I would probably try MDv4, MDv5a, and MDv5b on a couple of test datasets to see which works best, since it's a bit out of domain for all of them. But working is working.

[agentmorris]

Also for some of these edge cases, I've increasingly been using YOLOv5's test-time augmentation, which you can use with MegaDetector via run_inference_with_yolov5_val.py. If you have any videos where you can't quite get MD to find some of these out-of-domain animals, give that a try, or if you want me to give it a try first (it's still a little experimental), let me know by email.

[aweaver1fandm]

Thanks for the fix. It is working now. Also thank you for the suggestion about YOLO5 test-time augmentation. We may give that a shot knowing it's experimental and if we run into issues we'll let you know. I also think part of the issue is the box itself. Since the model was probably trained exclusively on images from nature, the box as a background probably doesn't help