Closed Antorithms closed 1 year ago
Hi Timo, Just adding one detail, during more testing today we noticed that right before crashing at recording stop there is a big spike to 100% in cuda usage. While recording cuda usage is at 50%. All of this observed from task manager.
Hi Timo, Thanks a lot for the updates that fixed the crashes. We stressed-test the system a bit to match our requirements and noticed a few things.
Thanks a lot for your help and again for the amazing tool !!
Attached are two example metadata files with the reported behavior. metadata11.csv metadata.csv
Dear Timo, Thanks a lot again for your support.
Thanks again for your kind help.
Good to hear you figured out a solution that works for you!
I think it should be possible to identify dropped frames reliably using the frame_id column in the metadata. If a camera drops a frame the frame_id will still be incremented on the camera. This means that you know that any a camera with frame_ids ...,13,14,17,18,19,... has dropped frames 15 and 16.
You can use those videos in the rest of the JARVIS pipeline as long as you correct the videos to be synchronized and have the same length. I would suggest to do this by adding additional frames (either black ones, or repeats of the ones before) to 'replace' the dropped ones and restore synchronization. I will try to add functionality to do this to the AnnotationTool in the next release, or at the very least upload a script that does it automatically.
Please let me know if you have any more questions or need any help getting the videos synchronized. I'll close this issue for now but feel free to reopen it at any point in time!
Hi Timo,
We are still working to get the system up to speed for six cameras. Just to recap: we have 6 BFS-U3-32S4M cameras and a computer with an i7-9800X, an RTX 3080, an M2 SSD, 2PCiE Fresco 4xUSB cards. All below is doe with We did some progressive testing and were able to record 4 cameras at fullres, JPEG95 compression and we got synchronized videos of the same n of frames. Here we always get the error below error in the command window upon start. Additionally, sometimes upon stop of recording the acquisition tool crashes.
With 5 or 6 cameras we are not able to record at full res without the tool crashing immediately at start. Upon binning we can occasionally record with 5 or 6 cameras but either the tool crashes at start time or when we stop the recordings.
Finally and probably most imporantly, we tried to compress the data more (JPEG50) and the acquisition starts with 6 cameras at full resolution (same error as above) and works (tested up to 10 minutes more or less) but upon stopping the AcqTool freezes and the command window shows:
Sorry for the multilayered issue! Let us know what you think about these issues and thanks a lot for all the help and the nice repo and software!