search

raspberrypi / libcamera

Other
196 stars 74 forks source link

Libcamerasrc doesnt work when running pipeline directly from C++ code instead of gst-launch #150

Closed tcikel closed 2 weeks ago

tcikel commented 3 weeks ago

Hi,

I have a C++ code from which i am running this pipeline:


   std::string pipeline_str = "libcamerasrc camera-name=/base/soc/i2c0mux/i2c@0/imx708@1a zoom=0 name=daycam "
                                "! capsfilter caps=video/x-raw,width=1280,height=720,format=NV12 ! videoconvert ! v4l2h264enc "
                              "! video/x-h264,level=(string)4.2 ! h264parse ! rtspclientsink location=" + rtsp_location + " latency=" + std::to_string(latency);

After i start the pipeline i can see these errors from libcamera: 

[0:41:32.794787193] [3020]  WARN IPAManager ipa_manager.cpp:114 Public key not valid
[0:41:32.799375860] [3020]  INFO Camera camera_manager.cpp:313 libcamera v0.3.0+65-6ddd79b5
[0:41:32.958432650] [3023]  INFO RPI vc4.cpp:400 Registered camera /base/soc/i2c0mux/i2c@0/imx708@1a to Unicam device /dev/media5 and ISP device /dev/media4
[0:41:32.969842420] [3029]  INFO Camera camera.cpp:1183 configuring streams: (0) 1280x720-NV12
[0:41:32.972070637] [3023]  INFO RPI vc4.cpp:569 Sensor: /base/soc/i2c0mux/i2c@0/imx708@1a - Selected sensor format: 1536x864-SBGGR10_1X10 - Selected unicam format: 1536x864-pBAA
[0:41:35.005125039] [3023] ERROR IPCPipe ipc_pipe_unixsocket.cpp:134 Call timeout!
[0:41:35.005446578] [3023] ERROR IPCPipe ipc_pipe_unixsocket.cpp:66 Failed to call sync
[0:41:35.005504279] [3023] ERROR IPAProxy raspberrypi_ipa_proxy.cpp:261 Failed to call start
[0:41:35.016592734] [3023] ERROR IPCUnixSocket ipc_unixsocket.cpp:191 Failed to send: Connection refused
[0:41:35.016746356] [3023] ERROR IPCPipe ipc_pipe_unixsocket.cpp:66 Failed to call sync
[0:41:35.016991010] [3023] ERROR IPAProxy raspberrypi_ipa_proxy.cpp:432 Failed to call mapBuffers
[0:41:35.018025048] [3023] ERROR IPCUnixSocket ipc_unixsocket.cpp:191 Failed to send: Transport endpoint is not connected
[0:41:35.018218371] [3023] ERROR IPCPipe ipc_pipe_unixsocket.cpp:66 Failed to call sync
[0:41:35.018341272] [3023] ERROR IPAProxy raspberrypi_ipa_proxy.cpp:432 Failed to call mapBuffers
[0:41:35.170799489] [3023] ERROR IPCUnixSocket ipc_unixsocket.cpp:191 Failed to send: Transport endpoint is not connected
[0:41:35.171094011] [3023] ERROR IPCPipe ipc_pipe_unixsocket.cpp:80 Failed to call async
[0:41:35.171220708] [3023] ERROR IPAProxy raspberrypi_ipa_proxy.cpp:508 Failed to call prepareIsp

When i run the exact same pipeline from terminal using gst-launch everything runs properly and i cant see any issues when I enable debug for gstreamer with the gstreamer side of my code. This works fine: gst-launch-1.0 libcamerasrc camera-name=/base/soc/i2c0mux/i2c@0/imx708@1a zoom=0 name=daycam ! capsfilter caps=video/x-raw,width=1280,height=720,format=NV12 ! videoconvert ! v4l2h264enc ! video/x-h264,level=\(string\)4.2 ! h264parse ! rtspclientsink location=rtsp://localhost:8554/daystream latency=1000

But when I look into debug part of libcamera it seems that for some reason the pipeline from C++ does not load all the ipa modules: 


[0:42:05.890752609] [3046]  WARN IPAManager ipa_manager.cpp:114 Public key not valid
[0:42:05.892776118] [3046] DEBUG IPAModule ipa_module.cpp:334 ipa_rkisp1.so: IPA module /usr/local/lib/aarch64-linux-gnu/libcamera/ipa_rkisp1.so is signed
[0:42:05.893039697] [3046] DEBUG IPAManager ipa_manager.cpp:245 Loaded IPA module '/usr/local/lib/aarch64-linux-gnu/libcamera/ipa_rkisp1.so'
[0:42:05.894712594] [3046] DEBUG IPAModule ipa_module.cpp:334 ipa_rpi_vc4.so: IPA module /usr/local/lib/aarch64-linux-gnu/libcamera/ipa_rpi_vc4.so is signed
[0:42:05.894942286] [3046] DEBUG IPAManager ipa_manager.cpp:245 Loaded IPA module '/usr/local/lib/aarch64-linux-gnu/libcamera/ipa_rpi_vc4.so'
[0:42:05.895268474] [3046] DEBUG IPAModule ipa_module.cpp:334 ipa_soft_simple.so: IPA module /usr/local/lib/aarch64-linux-gnu/libcamera/ipa_soft_simple.so is signed
[0:42:05.895390819] [3046] DEBUG IPAManager ipa_manager.cpp:245 Loaded IPA module '/usr/local/lib/aarch64-linux-gnu/libcamera/ipa_soft_simple.so'
[0:42:05.895781059] [3046]  INFO Camera camera_manager.cpp:313 libcamera v0.3.0+65-6ddd79b5
[0:42:05.896200741] [3049] DEBUG Camera camera_manager.cpp:69 Starting camera manager
[0:42:05.896994220] [3049] DEBUG DeviceEnumerator device_enumerator.cpp:230 New media device "bcm2835-isp" created from /dev/media4
[0:42:05.897409366] [3049] DEBUG DeviceEnumerator device_enumerator.cpp:258 Added device /dev/media4: bcm2835-isp
[0:42:05.897771977] [3049] DEBUG DeviceEnumerator device_enumerator.cpp:230 New media device "bcm2835-isp" created from /dev/media2
[0:42:05.898019964] [3049] DEBUG DeviceEnumerator device_enumerator.cpp:258 Added device /dev/media2: bcm2835-isp
[0:42:05.898179142] [3049] DEBUG DeviceEnumerator device_enumerator.cpp:230 New media device "rpivid" created from /dev/media0
[0:42:05.898311487] [3049] DEBUG DeviceEnumerator device_enumerator.cpp:258 Added device /dev/media0: rpivid
[0:42:05.898465090] [3049] DEBUG DeviceEnumerator device_enumerator.cpp:230 New media device "unicam" created from /dev/media5

When i run the pipeline with gst-launch i dont see the pubky warning and much more ipa modules are loaded: 

[0:46:54.352857525] [3066] DEBUG IPAModule ipa_module.cpp:334 ipa_rkisp1.so: IPA module /usr/lib/aarch64-linux-gnu/libcamera/ipa_rkisp1.so is signed
[0:46:54.352989242] [3066] DEBUG IPAManager ipa_manager.cpp:245 Loaded IPA module '/usr/lib/aarch64-linux-gnu/libcamera/ipa_rkisp1.so'
[0:46:54.353144329] [3066] DEBUG IPAModule ipa_module.cpp:334 ipa_rpi_pisp.so: IPA module /usr/lib/aarch64-linux-gnu/libcamera/ipa_rpi_pisp.so is signed
[0:46:54.353201994] [3066] DEBUG IPAManager ipa_manager.cpp:245 Loaded IPA module '/usr/lib/aarch64-linux-gnu/libcamera/ipa_rpi_pisp.so'
[0:46:54.353351340] [3066] DEBUG IPAModule ipa_module.cpp:334 ipa_rpi_vc4.so: IPA module /usr/lib/aarch64-linux-gnu/libcamera/ipa_rpi_vc4.so is signed
[0:46:54.353406690] [3066] DEBUG IPAManager ipa_manager.cpp:245 Loaded IPA module '/usr/lib/aarch64-linux-gnu/libcamera/ipa_rpi_vc4.so'
[0:46:54.353497446] [3066] DEBUG IPAModule ipa_module.cpp:334 ipa_soft_simple.so: IPA module /usr/lib/aarch64-linux-gnu/libcamera/ipa_soft_simple.so is signed
[0:46:54.353540944] [3066] DEBUG IPAManager ipa_manager.cpp:245 Loaded IPA module '/usr/lib/aarch64-linux-gnu/libcamera/ipa_soft_simple.so'
[0:46:54.353617756] [3066] DEBUG IPAModule ipa_module.cpp:334 ipa_vimc.so: IPA module /usr/lib/aarch64-linux-gnu/libcamera/ipa_vimc.so is signed
[0:46:54.353657273] [3066] DEBUG IPAManager ipa_manager.cpp:245 Loaded IPA module '/usr/lib/aarch64-linux-gnu/libcamera/ipa_vimc.so'
[0:46:54.353714419] [3066]  INFO Camera camera_manager.cpp:313 libcamera v0.3.0+65-6ddd79b5
[0:46:54.353959910] [3069] DEBUG Camera camera_manager.cpp:69 Starting camera manager
[0:46:54.368593367] [3069] DEBUG DeviceEnumerator device_enumerator.cpp:230 New media device "uvcvideo" created from /dev/media3
[0:46:54.368664419] [3069] DEBUG DeviceEnumerator device_enumerator_udev.cpp:96 Defer media device /dev/media3 due to 2 missing dependencies
[0:46:54.369366819] [3069] DEBUG DeviceEnumerator device_enumerator_udev.cpp:322 All dependencies for media device /dev/media3 found

I have added one change into the libcamerasrc and that is I have added zoom property but since it works as its supposed to from gst-launch i dont think that my changes are causing this issue.

naushir commented 3 weeks ago 
>[0:41:32.794787193] [3020]  WARN IPAManager ipa_manager.cpp:114 Public key not valid

Your IPA errors suggest that you have some missing dependencies. Can you run sudo apt install -y libgnutls28-dev openssl and then reconfigure and rebuild libcamera.

I suspect the difference between your runs is that one is trying to use a local install of libcamera, and the other is using the system package installed libraries.

naushir commented 2 weeks ago

Any updates on this? If not, I'll close shortly.