[OTHER] *OR BUG Picamera 0.3.19-1 hangs on easy_capture script while older versions work fine

monkeymademe commented 5 months ago

I don't want to claim this is a bug cause its not really giving much information

I updated my system today to the latest picamera2 (0.3.19-1) and libcamera v0.2.0+120-eb00c13d - suddenly my v3 cameras stopped working in my script when they worked before.

Its a script that starts a stream and it just hangs not even allowing keyboard interrupts. In an effort to see if my cameras are toast. I used the easy_capture script to see if it works.

This is the output from terminal

Python 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from picamera2 import Picamera2
>>> Picamera2.set_logging(Picamera2.DEBUG)
>>> picam2 = Picamera2()
[0:02:04.622899557] [2013]  INFO Camera camera_manager.cpp:284 libcamera v0.2.0+120-eb00c13d
[0:02:04.635558974] [2016]  INFO RPI pisp.cpp:695 libpisp version v1.0.6 b567f0455680 17-06-2024 (10:20:00)
[0:02:04.757693176] [2016]  INFO RPI pisp.cpp:1154 Registered camera /base/axi/pcie@120000/rp1/i2c@88000/imx708@1a to CFE device /dev/media1 and ISP device /dev/media0 using PiSP variant BCM2712_C0
[0:02:04.759022655] [2013]  INFO Camera camera_manager.cpp:284 libcamera v0.2.0+120-eb00c13d
[0:02:04.771072906] [2023]  INFO RPI pisp.cpp:695 libpisp version v1.0.6 b567f0455680 17-06-2024 (10:20:00)
[0:02:04.924577790] [2023]  INFO RPI pisp.cpp:1154 Registered camera /base/axi/pcie@120000/rp1/i2c@88000/imx708@1a to CFE device /dev/media1 and ISP device /dev/media0 using PiSP variant BCM2712_C0
picamera2.picamera2 INFO: Initialization successful.
picamera2.picamera2 INFO: Camera now open.
picamera2.picamera2 DEBUG: <libcamera._libcamera.CameraManager object at 0x7fff1fca93f0>
>>> picam2.start_and_capture_file("test.jpg")
[0:02:10.324684140] [2013]  WARN V4L2 v4l2_pixelformat.cpp:344 Unsupported V4L2 pixel format RPBP
picamera2.picamera2 DEBUG: Requesting configuration: {'use_case': 'preview', 'buffer_count': 4, 'transform': <libcamera.Transform 'identity'>, 'display': 'main', 'encode': 'main', 'colour_space': <libcamera.ColorSpace 'sYCC'>, 'controls': <Controls: {'NoiseReductionMode': <NoiseReductionModeEnum.Minimal: 3>, 'FrameDurationLimits': (100, 83333)}>, 'main': {'size': (640, 480), 'format': 'XBGR8888', 'stride': 2560, 'framesize': 1228800}, 'lores': None, 'raw': {'size': (1536, 864), 'format': 'BGGR_PISP_COMP1', 'stride': 1536, 'framesize': 1327104}, 'queue': True, 'sensor': {'bit_depth': 10, 'output_size': (1536, 864)}}
picamera2.picamera2 INFO: Camera configuration has been adjusted!
[0:02:10.326091635] [2013]  INFO Camera camera.cpp:1183 configuring streams: (0) 640x480-XBGR8888 (1) 1536x864-BGGR_PISP_COMP1
[0:02:10.326338868] [2023]  INFO RPI pisp.cpp:1450 Sensor: /base/axi/pcie@120000/rp1/i2c@88000/imx708@1a - Selected sensor format: 1536x864-SBGGR10_1X10 - Selected CFE format: 1536x864-PC1B
picamera2.picamera2 INFO: Configuration successful!
picamera2.picamera2 DEBUG: Final configuration: {'use_case': 'preview', 'buffer_count': 4, 'transform': <libcamera.Transform 'identity'>, 'display': 'main', 'encode': 'main', 'colour_space': <libcamera.ColorSpace 'sYCC'>, 'controls': <Controls: {'NoiseReductionMode': <NoiseReductionModeEnum.Minimal: 3>, 'FrameDurationLimits': (100, 83333)}>, 'main': {'size': (640, 480), 'format': 'XBGR8888', 'stride': 2560, 'framesize': 1228800}, 'lores': None, 'raw': {'size': (1536, 864), 'format': 'BGGR_PISP_COMP1', 'stride': 1536, 'framesize': 1327104}, 'queue': True, 'sensor': {'bit_depth': 10, 'output_size': (1536, 864)}}
picamera2.picamera2 DEBUG: Streams: {'main': <libcamera._libcamera.Stream object at 0x7fff0d9c7270>, 'lores': None, 'raw': <libcamera._libcamera.Stream object at 0x7fff0d9c6970>}
picamera2 DEBUG: Allocated 4 buffers for stream 0 with fds [35, 38, 41, 44]
picamera2 DEBUG: Allocated 4 buffers for stream 1 with fds [47, 50, 53, 56]

After this it just stopped... So I tried again with a v2 camera and got this

Python 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from picamera2 import Picamera2
>>> picam2 = Picamera2()
[0:11:06.224905392] [2043]  INFO Camera camera_manager.cpp:284 libcamera v0.2.0+120-eb00c13d
[0:11:06.236794684] [2049]  INFO RPI pisp.cpp:695 libpisp version v1.0.6 b567f0455680 17-06-2024 (10:20:00)
[0:11:06.272370505] [2049]  INFO RPI pisp.cpp:1154 Registered camera /base/axi/pcie@120000/rp1/i2c@88000/imx219@10 to CFE device /dev/media0 and ISP device /dev/media2 using PiSP variant BCM2712_C0
[0:11:06.274088491] [2043]  INFO Camera camera_manager.cpp:284 libcamera v0.2.0+120-eb00c13d
[0:11:06.285950728] [2056]  INFO RPI pisp.cpp:695 libpisp version v1.0.6 b567f0455680 17-06-2024 (10:20:00)
[0:11:06.319488755] [2056]  INFO RPI pisp.cpp:1154 Registered camera /base/axi/pcie@120000/rp1/i2c@88000/imx219@10 to CFE device /dev/media0 and ISP device /dev/media2 using PiSP variant BCM2712_C0
>>> picam2.start_and_capture_file("test.jpg")
[0:11:12.491131810] [2043]  WARN V4L2 v4l2_pixelformat.cpp:344 Unsupported V4L2 pixel format RPBP
[0:11:12.492143403] [2043]  INFO Camera camera.cpp:1183 configuring streams: (0) 640x480-XBGR8888 (1) 640x480-BGGR_PISP_COMP1
[0:11:12.492376343] [2056]  INFO RPI pisp.cpp:1450 Sensor: /base/axi/pcie@120000/rp1/i2c@88000/imx219@10 - Selected sensor format: 640x480-SBGGR10_1X10 - Selected CFE format: 640x480-PC1B
Exception in thread Thread-2 (thread_func):
Traceback (most recent call last):
  File "/usr/lib/python3.11/threading.py", line 1038, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.11/threading.py", line 975, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3/dist-packages/picamera2/previews/null_preview.py", line 29, in thread_func
    callback(picam2)
  File "/usr/lib/python3/dist-packages/picamera2/previews/drm_preview.py", line 83, in handle_request
    picam2.process_requests(self)
  File "/usr/lib/python3/dist-packages/picamera2/picamera2.py", line 1281, in process_requests
    display.render_request(display_request)
  File "/usr/lib/python3/dist-packages/picamera2/previews/drm_preview.py", line 74, in render_request
    self.render_drm(self.picam2, completed_request)
  File "/usr/lib/python3/dist-packages/picamera2/previews/drm_preview.py", line 158, in render_drm
    raise RuntimeError("Failed to reserve DRM plane")
RuntimeError: Failed to reserve DRM plane

Finally I dug out an older but working picamera2 I had on another sdcard and running the same code (without the debug) this is 0.3.17-1

Python 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from picamera2 import Picamera2
>>> picam2 = Picamera2()
[0:07:19.986709186] [3487]  INFO Camera camera_manager.cpp:284 libcamera v0.2.0+46-075b54d5
[0:07:20.021358229] [3490]  WARN RPiSdn sdn.cpp:39 Using legacy SDN tuning - please consider moving SDN inside rpi.denoise
[0:07:20.023639494] [3490]  INFO RPI vc4.cpp:447 Registered camera /base/soc/i2c0mux/i2c@1/imx708@1a to Unicam device /dev/media1 and ISP device /dev/media0
[0:07:20.023715399] [3490]  INFO RPI pipeline_base.cpp:1144 Using configuration file '/usr/share/libcamera/pipeline/rpi/vc4/rpi_apps.yaml'
[0:07:20.026278545] [3487]  INFO Camera camera_manager.cpp:284 libcamera v0.2.0+46-075b54d5
[0:07:20.061569810] [3493]  WARN RPiSdn sdn.cpp:39 Using legacy SDN tuning - please consider moving SDN inside rpi.denoise
[0:07:20.063817391] [3493]  INFO RPI vc4.cpp:447 Registered camera /base/soc/i2c0mux/i2c@1/imx708@1a to Unicam device /dev/media1 and ISP device /dev/media0
[0:07:20.063891741] [3493]  INFO RPI pipeline_base.cpp:1144 Using configuration file '/usr/share/libcamera/pipeline/rpi/vc4/rpi_apps.yaml'
>>> picam2.start_and_capture_file("test.jpg")
[0:07:25.907053470] [3487]  INFO Camera camera.cpp:1183 configuring streams: (0) 640x480-XBGR8888 (1) 1536x864-SBGGR10_CSI2P
[0:07:25.908055810] [3493]  INFO RPI vc4.cpp:611 Sensor: /base/soc/i2c0mux/i2c@1/imx708@1a - Selected sensor format: 1536x864-SBGGR10_1X10 - Selected unicam format: 1536x864-pBAA
commit failed
commit failed
[0:07:27.071959931] [3497]  INFO Camera camera.cpp:1183 configuring streams: (0) 4608x2592-BGR888 (1) 4608x2592-SBGGR10_CSI2P
[0:07:27.076532361] [3493]  INFO RPI vc4.cpp:611 Sensor: /base/soc/i2c0mux/i2c@1/imx708@1a - Selected sensor format: 4608x2592-SBGGR10_1X10 - Selected unicam format: 4608x2592-pBAA
[0:07:27.870210299] [3497]  INFO Camera camera.cpp:1183 configuring streams: (0) 640x480-XBGR8888 (1) 1536x864-SBGGR10_CSI2P
[0:07:27.875903895] [3493]  INFO RPI vc4.cpp:611 Sensor: /base/soc/i2c0mux/i2c@1/imx708@1a - Selected sensor format: 1536x864-SBGGR10_1X10 - Selected unicam format: 1536x864-pBAA

So it worked and I got my test.jpg where the other times in 0.3.19 i did not? Or is it the latest libcamera?...

monkeymademe commented 5 months ago

Also I am not sure if its something I am missing or did not do or should update?

davidplowman commented 5 months ago

Hi, I've just tried the following

from picamera2 import Picamera2
picam2 = Picamera2()
picam2.start_and_capture_file("test.jpg")

and it seemed OK. Can you fill me in on the details of your system. What kind of a Pi is it? How did you update it? Are you connected directly with a screen and keyboard or remotely, via ssh or vnc? Also, does rpicam-hello (and the other rpicam apps) work? You could also try downgrading picamera2 using pip (which will hide the apt version), though I would generally delete it after the experiment to avoid getting muddled about which one it's using.

Thanks!

monkeymademe commented 5 months ago

System:

Pi5 - Debian GNU/Linux 12 (bookworm) x2 tested on 2 systems one was already updated when I noticed the issue the other was working and then updated then it stopped working Picamera2 0.3.19-1 and libcamera v0.2.0+120-eb00c13d

Pi4 - Debian GNU/Linux 12 (bookworm) Picamera2 0.3.17-1 and libcamera v0.2.0+46-075b54d5

Update method:

sudo apt update

Connection - SSH from the report above but since you mentioned it I went back to the pi5 and ran the code on desktop the QTgiPreview window opens but no content is shown its blank.

The pi4 running the same code was over ssh and I got my test.jpg

davidplowman commented 5 months ago

I'm getting a bit confused here, are we saying your Pi 4 is working and your Pi 5 isn't?

Also note that the recommended update procedure is

sudo apt update
sudo apt full-upgrade

I've run this all again on a Pi 5 which works for me. I have Picamera2 0.3.19. rpicam-hello --version reports

rpicam-apps build: 49344f2a8d18 17-06-2024 (12:09:08)
libcamera build: v0.3.0+65-6ddd79b5

and the firmware version (vcgencmd version) is

2024/04/20 11:53:30 
Copyright (c) 2012 Broadcom
version d1744d21 (release) (embedded)

and the Linux kernel (uname -a)

Linux raspberrypi 6.6.31+rpt-rpi-2712 #1 SMP PREEMPT Debian 1:6.6.31-1+rpt1 (2024-05-29) aarch64 GNU/Linux

so those might all be worth checking.

monkeymademe commented 5 months ago

Oh man I am so sorry... Total pebcak moment here. sudo apt full-upgrade was the issue and fixed the problem half way. But yeah I did not run that before.

After the webcam UI I am working on is working again so YAY... BUT...

Running the easy_capture in terminal on desktop the QtGIPreview window opens but still no content and is just black and hangs there. In SSH : the output is

Python 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from picamera2 import Picamera2
>>> picam2 = Picamera2()
[0:03:09.308991256] [3538]  INFO Camera camera_manager.cpp:313 libcamera v0.3.0+65-6ddd79b5
[0:03:09.317127318] [3541]  INFO RPI pisp.cpp:695 libpisp version v1.0.6 b567f0455680 17-06-2024 (10:20:00)
[0:03:09.410094418] [3541]  INFO RPI pisp.cpp:1154 Registered camera /base/axi/pcie@120000/rp1/i2c@88000/imx708@1a to CFE device /dev/media1 and ISP device /dev/media0 using PiSP variant BCM2712_C0
[0:03:09.411833601] [3538]  INFO Camera camera_manager.cpp:313 libcamera v0.3.0+65-6ddd79b5
[0:03:09.420133056] [3544]  INFO RPI pisp.cpp:695 libpisp version v1.0.6 b567f0455680 17-06-2024 (10:20:00)
[0:03:09.546696169] [3544]  INFO RPI pisp.cpp:1154 Registered camera /base/axi/pcie@120000/rp1/i2c@88000/imx708@1a to CFE device /dev/media1 and ISP device /dev/media0 using PiSP variant BCM2712_C0
>>> picam2.start_and_capture_file("test.jpg")
[0:03:13.709175917] [3538]  WARN V4L2 v4l2_pixelformat.cpp:344 Unsupported V4L2 pixel format RPBP
[0:03:13.710166610] [3538]  INFO Camera camera.cpp:1183 configuring streams: (0) 640x480-XBGR8888 (1) 1536x864-BGGR_PISP_COMP1
[0:03:13.710367504] [3544]  INFO RPI pisp.cpp:1450 Sensor: /base/axi/pcie@120000/rp1/i2c@88000/imx708@1a - Selected sensor format: 1536x864-SBGGR10_1X10 - Selected CFE format: 1536x864-PC1B
Exception in thread Thread-2 (thread_func):
Traceback (most recent call last):
  File "/usr/lib/python3.11/threading.py", line 1038, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.11/threading.py", line 975, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3/dist-packages/picamera2/previews/null_preview.py", line 29, in thread_func
    callback(picam2)
  File "/usr/lib/python3/dist-packages/picamera2/previews/drm_preview.py", line 83, in handle_request
    picam2.process_requests(self)
  File "/usr/lib/python3/dist-packages/picamera2/picamera2.py", line 1281, in process_requests
    display.render_request(display_request)
  File "/usr/lib/python3/dist-packages/picamera2/previews/drm_preview.py", line 74, in render_request
    self.render_drm(self.picam2, completed_request)
  File "/usr/lib/python3/dist-packages/picamera2/previews/drm_preview.py", line 158, in render_drm
    raise RuntimeError("Failed to reserve DRM plane")
RuntimeError: Failed to reserve DRM plane

For now I guess its working enough for me to continue what I am doing but I don't know what the issue is with the black preview window (camera works fine otherwise)

davidplowman commented 5 months ago

Well, there's still something a bit weird going on there. If it's complaining about DRM, that means it thinks the desktop isn't running and that it's not connected via ssh. To be fair, figuring out how you're connected is actually quite tricky, so that can go wrong. It would certainly get confused if you use ssh but without -X (X-forwarding) and then try to display a preview window. So maybe just check what the Pi's $DISPLAY variable says once you've logged in. The rules are:

ssh without -X : you can't have a preview window.
ssh with -X : preview window should work, but you'll get a software rendered preview spamming your network with uncompressed video.

monkeymademe commented 5 months ago

Yeah checked export but display is not set... then for giggles I retested the easy_capture and it worked fine! Might have needed a reboot and a dodgy ssh connection... new computer and its not totally setup the way I like it yet.

Thank you so much for the support and sorry it was a silly one

davidplowman commented 5 months ago

No problem, good luck!!

raspberrypi / picamera2

[OTHER] *OR BUG Picamera 0.3.19-1 hangs on easy_capture script while older versions work fine #1059