Bonsai gives empty output file

[JohnNan123]

Hi all,

This is Junyu. Anshuman and I have the Bonsai installed and it is running without any bugs popping up. Yet the problem is it does not export any metadata nor videos (neither streaming windows pop up as shown in the tutorial nor recordings to the root folder). Moreover, it created a folder at the root folder and 6 CSV files named by the serial number, yet each file remains empty. There is no video files created and the cmd prompt does not report anything.

[bruno-f-cruz]

Hi!, Are the cameras acquiring frames? Are you triggering them?

[JohnNan123]

This might be a dumb question but how do I know if the camera is acquiring frames?

[JohnNan123]

Do we need to add some additional functions to the workflow? so far we just have the camera there as you provided us in main.bonsai and clicked start

[bruno-f-cruz]

While bonsai is running, double click Camera yellow node. IT should show you that the camera is streaming. If it is white, means the camera is not sending out data but it is connected (E.g. waiting for a trigger)

[JohnNan123]

It is the same situation as you explained. It have a white window poped up

[bruno-f-cruz]

Sounds like the camera is not being triggered then. Check with spinview first!

[JohnNan123]

at spinview it has the trigger mode on, and not acquire any frames. May I know how to trigger them in Bonsai? I searched at the toolbox and it gives me 3 options: TriggeredBugger, BufferTrigger and WindowTrigger

[bruno-f-cruz]

I thought you had a script to trigger the camera via an arduino? Bonsai is not triggering the camera at all. If you don't want to trigger the camera, just set it to continuous mode.

[JohnNan123]

Oh I see. That script is implemented inside campy and we will need some time to isolate that function. But thank you for your explaination

[JohnNan123]

Hi Bruno,

I have turned off the trigger mode and run the workflow. the window shows the image for like 5 seconds and popped up an error message (runtime error) that says pipe is broken.

[bruno-f-cruz]

What is the output of the bonsai console (black terminal window)? I suspect ffmpeg is raising an error.

[JohnNan123]

The terminal is black. I believe ffmpeg is causing that problem too but not sure exactly how.

[bruno-f-cruz]

What happens if you call ffmpeg from the command line? Also what gpu are you using?

[JohnNan123]

Hi Bruno,

Happy New Year! Sorry for the late reply. the GPU we are using is RTX4000. When you mentioned about calling ffmpeg, do you mean just calling the ffmpeg? or the one we sent you before with all the options. Here is the output from the terminal.

(base) PS C:\Users\as1296> ffmpeg
ffmpeg version 2023-12-18-git-be8a4f80b9-full_build-www.gyan.dev Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 12.2.0 (Rev10, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --pkg-config=pkgconf --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libaribb24 --enable-libaribcaption --enable-libdav1d --enable-libdavs2 --enable-libuavs3d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libaom --enable-libjxl --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable-liblensfun --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-dxva2 --enable-d3d11va --enable-libvpl --enable-libshaderc --enable-vulkan --enable-libplacebo --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libcodec2 --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
  libavutil      58. 34.100 / 58. 34.100
  libavcodec     60. 35.100 / 60. 35.100
  libavformat    60. 18.100 / 60. 18.100
  libavdevice    60.  4.100 / 60.  4.100
  libavfilter     9. 14.100 /  9. 14.100
  libswscale      7.  6.100 /  7.  6.100
  libswresample   4. 13.100 /  4. 13.100
  libpostproc    57.  4.100 / 57.  4.100
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...

Use -h to get full help or, even better, run 'man ffmpeg'
(base) PS C:\Users\as1296>

[JohnNan123]

Actually, I found the error message output from the Bonsai terminal. Hope this can help

ffmpeg version [2023-12-18-git-be8a4f80b9-full_build-www.gyan.dev](http://2023-12-18-git-be8a4f80b9-full_build-www.gyan.dev/) Copyright (c) 2000-2023 the FFmpeg developers
ffmpeg version [2023-12-18-git-be8a4f80b9-full_build-www.gyan.dev](http://2023-12-18-git-be8a4f80b9-full_build-www.gyan.dev/) Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 12.2.0 (Rev10, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --pkg-config=pkgconf --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libaribb24 --enable-libaribcaption --enable-libdav1d --enable-libdavs2 --enable-libuavs3d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libaom --enable-libjxl --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable-liblensfun --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-dxva2 --enable-d3d11va --enable-libvpl --enable-libshaderc --enable-vulkan --enable-libplacebo --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libcodec2 --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
  libavutil      58. 34.100 / 58. 34.100
  libavcodec     60. 35.100 / 60. 35.100
  libavformat    60. 18.100 / 60. 18.100
  libavdevice    60.  4.100 / 60.  4.100
  libavfilter     9. 14.100 /  9. 14.100
  libswscale      7.  6.100 /  7.  6.100
  libswresample   4. 13.100 /  4. 13.100
  libpostproc    57.  4.100 / 57.  4.100
  built with gcc 12.2.0 (Rev10, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --pkg-config=pkgconf --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libaribb24 --enable-libaribcaption --enable-libdav1d --enable-libdavs2 --enable-libuavs3d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libaom --enable-libjxl --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable-liblensfun --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-dxva2 --enable-d3d11va --enable-libvpl --enable-libshaderc --enable-vulkan --enable-libplacebo --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libcodec2 --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
  libavutil      58. 34.100 / 58. 34.100
  libavcodec     60. 35.100 / 60. 35.100
  libavformat    60. 18.100 / 60. 18.100
  libavdevice    60.  4.100 / 60.  4.100
  libavfilter     9. 14.100 /  9. 14.100
  libswscale      7.  6.100 /  7.  6.100
  libswresample   4. 13.100 /  4. 13.100
  libpostproc    57.  4.100 / 57.  4.100
-vsync is deprecated. Use -fps_mode
Passing a number to -vsync is deprecated, use a string argument as described in the manual.
Input #0, rawvideo, from '\\.\pipe\21090111':
  Duration: N/A, start: 0.000000, bitrate: 2972712 kb/s
  Stream #0:0: Video: rawvideo (BGR[24] / 0x18524742), bgr24, 2304x1792, 2972712 kb/s, 30 tbr, 30 tbn
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (h264_nvenc))
Press [q] to stop, [?] for help
[h264_nvenc @ 0000021e3cca4380] Driver does not support the required nvenc API version. Required: 12.1 Found: 11.1
[h264_nvenc @ 0000021e3cca4380] The minimum required Nvidia driver for nvenc is (unknown) or newer
[vost#0:0/h264_nvenc @ 0000021e3cca3e80] Error while opening encoder - maybe incorrect parameters such as bit_rate, rate, width or height.
[vf#0:0 @ 0000021e3ccbf7c0] Error sending frames to consumers: Function not implemented
[vf#0:0 @ 0000021e3ccbf7c0] Task finished with error code: -40 (Function not implemented)
[vf#0:0 @ 0000021e3ccbf7c0] Terminating thread with return code -40 (Function not implemented)
[vost#0:0/h264_nvenc @ 0000021e3cca3e80] Could not open encoder before EOF
[vost#0:0/h264_nvenc @ 0000021e3cca3e80] Task finished with error code: -22 (Invalid argument)
[vost#0:0/h264_nvenc @ 0000021e3cca3e80] Terminating thread with return code -22 (Invalid argument)
[out#0/mp4 @ 0000021e3cc90a80] Nothing was written into output file, because at least one of its streams received no packets.
frame=    0 fps=0.0 q=0.0 Lsize=       0kB time=N/A bitrate=N/A speed=N/A
Conversion failed!
-vsync is deprecated. Use -fps_mode
Passing a number to -vsync is deprecated, use a string argument as described in the manual.
Input #0, rawvideo, from '\\.\pipe\21090109':
  Duration: N/A, start: 0.000000, bitrate: 2972712 kb/s
  Stream #0:0: Video: rawvideo (BGR[24] / 0x18524742), bgr24, 2304x1792, 2972712 kb/s, 30 tbr, 30 tbn
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (h264_nvenc))
Press [q] to stop, [?] for help
[h264_nvenc @ 000001e1844c4380] Driver does not support the required nvenc API version. Required: 12.1 Found: 11.1
[h264_nvenc @ 000001e1844c4380] The minimum required Nvidia driver for nvenc is (unknown) or newer
[vost#0:0/h264_nvenc @ 000001e1844c3e80] Error while opening encoder - maybe incorrect parameters such as bit_rate, rate, width or height.
[vf#0:0 @ 000001e1844df7c0] Error sending frames to consumers: Function not implemented
[vf#0:0 @ 000001e1844df7c0] Task finished with error code: -40 (Function not implemented)
[vf#0:0 @ 000001e1844df7c0] Terminating thread with return code -40 (Function not implemented)
[vost#0:0/h264_nvenc @ 000001e1844c3e80] Could not open encoder before EOF
[vost#0:0/h264_nvenc @ 000001e1844c3e80] Task finished with error code: -22 (Invalid argument)
[vost#0:0/h264_nvenc @ 000001e1844c3e80] Terminating thread with return code -22 (Invalid argument)
[out#0/mp4 @ 000001e1844b0a80] Nothing was written into output file, because at least one of its streams received no packets.
frame=    0 fps=0.0 q=0.0 Lsize=       0kB time=N/A bitrate=N/A speed=N/A
Conversion failed!

[JohnNan123]

The GPU version `NVIDIA System Information report created on: 01/03/2024 14:28:16 System name: DESKTOP-AVS3ITU

[Display] Operating System: Windows 10 Education, 64-bit DirectX version: 12.0 GPU processor: Quadro RTX 4000 Driver version: 472.84 Driver Type: Standard Direct3D feature level: 12_1 CUDA Cores: 2304 Core clock: 1545 MHz Memory data rate: 13.00 Gbps Memory interface: 256-bit Memory bandwidth: 416.06 GB/s Total available graphics memory: 139101 MB Dedicated video memory: 8192 MB GDDR6 System video memory: 0 MB Shared system memory: 130909 MB Video BIOS version: 90.04.87.00.01 IRQ: Not used Bus: PCI Express x16 Gen3 Device Id: 10DE 1EB1 12A010DE Part Number: G160 0500

[Components]

nvui.dll 8.17.14.7284 NVIDIA User Experience Driver Component nvxdplcy.dll 8.17.14.7284 NVIDIA User Experience Driver Component nvxdbat.dll 8.17.14.7284 NVIDIA User Experience Driver Component nvxdapix.dll 8.17.14.7284 NVIDIA User Experience Driver Component nvCplUIR.dll 8.1.940.0 NVIDIA Control Panel nvCplUI.exe 8.1.940.0 NVIDIA Control Panel nvWSSR.dll 30.0.14.7284 NVIDIA Workstation Server nvWSS.dll 30.0.14.7284 NVIDIA Workstation Server nvViTvSR.dll 30.0.14.7284 NVIDIA Video Server nvViTvS.dll 30.0.14.7284 NVIDIA Video Server nvLicensingS.dll 6.14.14.7284 NVIDIA Licensing Server nvDispSR.dll 30.0.14.7284 NVIDIA Display Server nvDispS.dll 30.0.14.7284 NVIDIA Display Server nvDevToolSR.dll 30.0.14.7284 NVIDIA Licensing Server nvDevToolS.dll 30.0.14.7284 NVIDIA 3D Settings Server NVCUDA64.DLL 30.0.14.7284 NVIDIA CUDA 11.4.176 driver nvGameSR.dll 30.0.14.7284 NVIDIA 3D Settings Server nvGameS.dll 30.0.14.7284 NVIDIA 3D Settings Server The CUDA version: I don't think our computer has installed CUDA but I can have it installed. and the ffmpeg version: (base) PS C:\Users\as1296> ffmpeg -version ffmpeg version 2023-12-18-git-be8a4f80b9-full_build-www.gyan.dev Copyright (c) 2000-2023 the FFmpeg developers built with gcc 12.2.0 (Rev10, Built by MSYS2 project) configuration: --enable-gpl --enable-version3 --enable-static --pkg-config=pkgconf --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libaribb24 --enable-libaribcaption --enable-libdav1d --enable-libdavs2 --enable-libuavs3d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libaom --enable-libjxl --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable-liblensfun --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-dxva2 --enable-d3d11va --enable-libvpl --enable-libshaderc --enable-vulkan --enable-libplacebo --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libcodec2 --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint libavutil 58. 34.100 / 58. 34.100 libavcodec 60. 35.100 / 60. 35.100 libavformat 60. 18.100 / 60. 18.100 libavdevice 60. 4.100 / 60. 4.100 libavfilter 9. 14.100 / 9. 14.100 libswscale 7. 6.100 / 7. 6.100 libswresample 4. 13.100 / 4. 13.100 libpostproc 57. 4.100 / 57. 4.100 `

[bruno-f-cruz]

Can you update the drivers of the GPU to the latest version and try again?

[JohnNan123]

[JohnNan123]

Hi Bruno,

I have uploaded the GPU driver to 537.99 and the bug is resolved! I have run a test streaming but found that there is a lagging of the camera image and also a little dis-synchronized. I have attached a video showing that. I have double-checked the metadata and videos. The video remains black which seems to image and been stored into it. The metadata shows the cameras are not synchronized perfectly.

https://github.com/bruno-f-cruz/hexa_flir_demo/assets/90344626/b707dd74-a035-4949-9041-3d96710e5a21

[bruno-f-cruz]

Synchronization is never guaranteed in software, you should take care of it using a common trigger to all cameras! How were you doing it before?

Can you record video for 5minutes and keep track of the used RAM? I am wondering if the gpu is being able to encode all videos in real time. Can you check the ffmpeg status in the bonsai command window?

[JohnNan123]

I have the bonsai terminal output copied down here. The size of the output video file varies a lot, and all of them remain black when I try to open them. this computer has 256g RAM, which should be enough?

ffmpeg output.txt

[bruno-f-cruz]

I want to see if the memory is accumulating during runtime not necessarily whether you have enough or not. Are all cameras the same? Do they all have the same settings? Is a common trigger being sent to all of them at the same time? I am not sure why would one expect the same frame in the two cameras to have the same timestamp. They are different cameras running on two distinct clocks. I might be missing something sorry...

[JohnNan123]

Hi Bruno,

Sorry for the confusion. I am not a camera expert and still learning it. Appoligize for the confusion.

All the cameras are the same and have the same setting (except for exposure time). They are triggered from the common port on the Audrino Board so they are expected to be triggered at the same time. I might be wrong with the timestamp but I observed from the streaming that the laser point from one camera disappears earlier than the other ones. I am trying to mark that they are not synchronized perfectly. It should be easier to solve when you have access to our computer than through text.

[bruno-f-cruz]

Do all files have the same number of frames? Do you see any dropped frames? Are you ensuring that the exposure setting of all cameras is compatible with the trigger frequency?