search

raspberrypi / picamera2

New libcamera based python library
BSD 2-Clause "Simplified" License
891 stars 188 forks source link

[EXAMPLES] Added more current model (Yolov5s) example to the tensorflow object detection lineup #965

Closed Zanz2 closed 8 months ago

Zanz2 commented 8 months ago

I added a Yolov5s example to the examples repository (including model and labels), because the inputs/outputs are quite different than mobilenet and detectnet the scripts had to be adjusted quite a bit, so i thought it warranted a new example. Its pretty self contained so there are no extra requirements (other than what you would need for the examples that already exist).

I tested this on the raspberry pi 5 with the camera module v3.

davidplowman commented 8 months ago

Hi, thanks for this example. It looks like a nice and simple self-contained demo, so it would be nice to merge it.

A few things to sort out first. Could you check the contribution guidelines? I don't think there will be too much, but I'm sure it will want some cosmetic changes. We should also target it at the "next" branch, and make sure it's signed off.

One extra thing to think about. The use of the RGB888 lores stream means it will only work on a Pi 5. Is there any way to make this work on (for example) a Pi 4? I guess maybe there's just too much computation in which case perhaps we could quit and make it clear what the problem is, maybe

from picamera2 import Platform

and then somewhere

    if Picamera2.platform != Platform.PISP:
        print("Sorry, only supported on Pi 5")
        sys.exit(-1)

or something like that?

Zanz2 commented 8 months ago

Ill read the guidelines, thanks! Thats an excellent suggestion! I added compatibility code so that it should work also on systems that only support YUV420, the colorspace is just converted to RGB888 inside the matrix with numpy/opencv! You could in theory use YUV420 on a pi5 but i suspect using rgb888 directly speeds it up a tiny bit because you dont have to convert it manually later.

Zanz2 commented 8 months ago

Hi, i fixed all the issues and made sure everything is according to the contribution guidelines.

These are my test results, the drm overlay doesnt work on my system, but im sure it has nothing to do with the example i added since it fails normally too.

picamera2/tests/test_list.txt']
Running 84 tests

Running tests in /home/zanza/picamera2/tests/test_list_drm.txt
Running /home/zanza/picamera2/examples/overlay_drm.py...    FAILED
====================    STDOUT    ====================
[78:03:15.950849183] [397953]  INFO Camera camera_manager.cpp:284 libcamera v0.2.0+46-075b54d5
    [78:03:15.961191002] [397958]  INFO RPI pisp.cpp:662 libpisp version v1.0.4 6e3a53d137f4 14-02-2024 (14:00:12)
    [78:03:15.971028838] [397958]  INFO RPI pisp.cpp:1121 Registered camera /base/axi/pcie@120000/rp1/i2c@88000/imx708@1a to CFE device /dev/media0 and ISP device /dev/media1 using PiSP variant BCM2712_C0
    [78:03:15.972371989] [397953]  INFO Camera camera_manager.cpp:284 libcamera v0.2.0+46-075b54d5
    [78:03:15.981477675] [397961]  INFO RPI pisp.cpp:662 libpisp version v1.0.4 6e3a53d137f4 14-02-2024 (14:00:12)
    [78:03:15.991434623] [397961]  INFO RPI pisp.cpp:1121 Registered camera /base/axi/pcie@120000/rp1/i2c@88000/imx708@1a to CFE device /dev/media0 and ISP device /dev/media1 using PiSP variant BCM2712_C0
    [78:03:15.994406370] [397953]  INFO Camera camera.cpp:1183 configuring streams: (0) 640x480-XBGR8888 (1) 1536x864-BGGR16_PISP_COMP1
    [78:03:15.994522148] [397961]  INFO RPI pisp.cpp:1405 Sensor: /base/axi/pcie@120000/rp1/i2c@88000/imx708@1a - Selected sensor format: 1536x864-SBGGR10_1X10 - Selected CFE format: 1536x864-PC1B
    Exception in thread Thread-2 (thread_func):
    Traceback (most recent call last):
      File "/usr/lib/python3.11/threading.py", line 1038, in _bootstrap_inner
        self.run()
      File "/usr/lib/python3.11/threading.py", line 975, in run
        self._target(*self._args, **self._kwargs)
      File "/usr/lib/python3/dist-packages/picamera2/previews/null_preview.py", line 29, in thread_func
        callback(picam2)
      File "/usr/lib/python3/dist-packages/picamera2/previews/drm_preview.py", line 83, in handle_request
        picam2.process_requests(self)
      File "/usr/lib/python3/dist-packages/picamera2/picamera2.py", line 1278, in process_requests
        display.render_request(display_request)
      File "/usr/lib/python3/dist-packages/picamera2/previews/drm_preview.py", line 74, in render_request
        self.render_drm(self.picam2, completed_request)
      File "/usr/lib/python3/dist-packages/picamera2/previews/drm_preview.py", line 153, in render_drm
        raise RuntimeError("Failed to reserve DRM plane")
    RuntimeError: Failed to reserve DRM plane
    Traceback (most recent call last):
      File "/home/zanza/picamera2/examples/overlay_drm.py", line 20, in <module>
        picam2.set_overlay(overlay)
      File "/usr/lib/python3/dist-packages/picamera2/picamera2.py", line 1781, in set_overlay
        self._preview.set_overlay(overlay)
      File "/usr/lib/python3/dist-packages/picamera2/previews/drm_preview.py", line 120, in set_overlay
        self.render_drm(self.picam2, None)
      File "/usr/lib/python3/dist-packages/picamera2/previews/drm_preview.py", line 153, in render_drm
        raise RuntimeError("Failed to reserve DRM plane")
    RuntimeError: Failed to reserve DRM plane

Running /home/zanza/picamera2/examples/preview_drm.py...    ERROR
         raise runtimeerror("failed to reserve drm plane")
Running /home/zanza/picamera2/tests/drm_stop_restart.py...  TIMED OUT
====================    STDOUT    ====================
[78:03:23.478478009] [397987]  INFO Camera camera_manager.cpp:284 libcamera v0.2.0+46-075b54d5
    [78:03:23.487299936] [397992]  INFO RPI pisp.cpp:662 libpisp version v1.0.4 6e3a53d137f4 14-02-2024 (14:00:12)
    [78:03:23.496893697] [397992]  INFO RPI pisp.cpp:1121 Registered camera /base/axi/pcie@120000/rp1/i2c@88000/imx708@1a to CFE device /dev/media0 and ISP device /dev/media1 using PiSP variant BCM2712_C0
    [78:03:23.498119904] [397987]  INFO Camera camera_manager.cpp:284 libcamera v0.2.0+46-075b54d5
    [78:03:23.506980145] [397995]  INFO RPI pisp.cpp:662 libpisp version v1.0.4 6e3a53d137f4 14-02-2024 (14:00:12)
    [78:03:23.516855333] [397995]  INFO RPI pisp.cpp:1121 Registered camera /base/axi/pcie@120000/rp1/i2c@88000/imx708@1a to CFE device /dev/media0 and ISP device /dev/media1 using PiSP variant BCM2712_C0
    [78:03:23.524566146] [397987]  INFO Camera camera.cpp:1183 configuring streams: (0) 640x480-XBGR8888 (1) 1536x864-BGGR16_PISP_COMP1
    [78:03:23.524695609] [397995]  INFO RPI pisp.cpp:1405 Sensor: /base/axi/pcie@120000/rp1/i2c@88000/imx708@1a - Selected sensor format: 1536x864-SBGGR10_1X10 - Selected CFE format: 1536x864-PC1B
    Exception in thread Thread-2 (thread_func):
    Traceback (most recent call last):
      File "/usr/lib/python3.11/threading.py", line 1038, in _bootstrap_inner
        self.run()
      File "/usr/lib/python3.11/threading.py", line 975, in run
        self._target(*self._args, **self._kwargs)
      File "/usr/lib/python3/dist-packages/picamera2/previews/null_preview.py", line 29, in thread_func
        callback(picam2)
      File "/usr/lib/python3/dist-packages/picamera2/previews/drm_preview.py", line 83, in handle_request
        picam2.process_requests(self)
      File "/usr/lib/python3/dist-packages/picamera2/picamera2.py", line 1278, in process_requests
        display.render_request(display_request)
      File "/usr/lib/python3/dist-packages/picamera2/previews/drm_preview.py", line 74, in render_request
        self.render_drm(self.picam2, completed_request)
      File "/usr/lib/python3/dist-packages/picamera2/previews/drm_preview.py", line 153, in render_drm
        raise RuntimeError("Failed to reserve DRM plane")
    RuntimeError: Failed to reserve DRM plane
    Exception in thread Thread-3 (thread_func):
    Traceback (most recent call last):
      File "/usr/lib/python3.11/threading.py", line 1038, in _bootstrap_inner
        self.run()
      File "/usr/lib/python3.11/threading.py", line 975, in run
        self._target(*self._args, **self._kwargs)
      File "/usr/lib/python3/dist-packages/picamera2/previews/null_preview.py", line 29, in thread_func
        callback(picam2)
      File "/usr/lib/python3/dist-packages/picamera2/previews/drm_preview.py", line 83, in handle_request
        picam2.process_requests(self)
      File "/usr/lib/python3/dist-packages/picamera2/picamera2.py", line 1278, in process_requests
        display.render_request(display_request)
      File "/usr/lib/python3/dist-packages/picamera2/previews/drm_preview.py", line 74, in render_request
        self.render_drm(self.picam2, completed_request)
      File "/usr/lib/python3/dist-packages/picamera2/previews/drm_preview.py", line 153, in render_drm
        raise RuntimeError("Failed to reserve DRM plane")
    RuntimeError: Failed to reserve DRM plane

Running tests in /home/zanza/picamera2/tests/test_list.txt
Running /home/zanza/picamera2/examples/capture_circular_nooutput.py...  PASSED
Running /home/zanza/picamera2/examples/capture_dng.py...    PASSED
Running /home/zanza/picamera2/examples/capture_dng_and_jpeg.py...   PASSED
Running /home/zanza/picamera2/examples/capture_dng_and_jpeg_helpers.py...   PASSED
Running /home/zanza/picamera2/examples/capture_full_res.py...   PASSED
Running /home/zanza/picamera2/examples/capture_headless.py...   PASSED
Running /home/zanza/picamera2/examples/capture_helpers.py...    PASSED
Running /home/zanza/picamera2/examples/capture_mjpeg.py...  PASSED
Running /home/zanza/picamera2/examples/capture_old_request.py...    PASSED
Running /home/zanza/picamera2/examples/capture_png.py...    PASSED
Running /home/zanza/picamera2/examples/capture_to_buffer.py...  PASSED
Running /home/zanza/picamera2/examples/capture_video.py...  PASSED
Running /home/zanza/picamera2/examples/capture_video_multiple.py...     PASSED
Running /home/zanza/picamera2/examples/capture_video_multiple_2.py...   PASSED
Running /home/zanza/picamera2/examples/controls.py...   PASSED
Running /home/zanza/picamera2/examples/controls_2.py...     PASSED
Running /home/zanza/picamera2/examples/display_transform_qtgl.py...     PASSED
Running /home/zanza/picamera2/examples/dual_encode.py...    PASSED
Running /home/zanza/picamera2/examples/easy_capture.py...   PASSED
Running /home/zanza/picamera2/examples/easy_video.py...     PASSED
Running /home/zanza/picamera2/examples/exif_custom.py...    PASSED
Running /home/zanza/picamera2/examples/exposure_fixed.py...     PASSED
Running /home/zanza/picamera2/examples/frame_server.py...   PASSED
Running /home/zanza/picamera2/examples/h264_constant_qp.py...   PASSED
Running /home/zanza/picamera2/examples/metadata.py...   PASSED
Running /home/zanza/picamera2/examples/mp4_capture.py...    PASSED
Running /home/zanza/picamera2/examples/opencv_face_detect_2.py...   PASSED
Running /home/zanza/picamera2/examples/opencv_mertens_merge.py...   PASSED
Running /home/zanza/picamera2/examples/overlay_gl.py...     PASSED
Running /home/zanza/picamera2/examples/overlay_null.py...   PASSED
Running /home/zanza/picamera2/examples/overlay_qt.py...     PASSED
Running /home/zanza/picamera2/examples/pick_mode.py...  PASSED
Running /home/zanza/picamera2/examples/preview.py...    PASSED
Running /home/zanza/picamera2/examples/preview_x_forwarding.py...   PASSED
Running /home/zanza/picamera2/examples/raw.py...    PASSED
Running /home/zanza/picamera2/examples/rotation.py...   PASSED
Running /home/zanza/picamera2/examples/still_capture_with_config.py...  PASSED
Running /home/zanza/picamera2/examples/still_during_video.py...     PASSED
Running /home/zanza/picamera2/examples/switch_mode.py...    PASSED
Running /home/zanza/picamera2/examples/switch_mode_2.py...  PASSED
Running /home/zanza/picamera2/examples/switch_mode_persist.py...    PASSED
Running /home/zanza/picamera2/examples/timestamp_capture.py...  PASSED
Running /home/zanza/picamera2/examples/title_bar.py...  PASSED
Running /home/zanza/picamera2/examples/tuning_file.py...    PASSED
Running /home/zanza/picamera2/examples/video_with_config.py...  PASSED
Running /home/zanza/picamera2/examples/window_offset.py...  PASSED
Running /home/zanza/picamera2/examples/zoom.py...   PASSED
Running /home/zanza/picamera2/tests/app_dual.py...  SKIPPED
Running /home/zanza/picamera2/tests/app_full_test.py...     PASSED
Running /home/zanza/picamera2/tests/app_test.py...  PASSED
Running /home/zanza/picamera2/tests/autofocus_test.py...    PASSED
Running /home/zanza/picamera2/tests/async_test.py...    PASSED
Running /home/zanza/picamera2/tests/bitrate_check.py...     PASSED
Running /home/zanza/picamera2/tests/check_timestamps.py...  PASSED
Running /home/zanza/picamera2/tests/close_test.py...    PASSED
Running /home/zanza/picamera2/tests/close_test_multiple.py...   SKIPPED
Running /home/zanza/picamera2/tests/codecs.py...    PASSED
Running /home/zanza/picamera2/tests/colour_spaces.py...     PASSED
Running /home/zanza/picamera2/tests/config_with_sensor.py...    PASSED
Running /home/zanza/picamera2/tests/configurations.py...    PASSED
Running /home/zanza/picamera2/tests/context_test.py...  PASSED
Running /home/zanza/picamera2/tests/display_transform_null.py...    PASSED
Running /home/zanza/picamera2/tests/display_transform_qt.py...  PASSED
Running /home/zanza/picamera2/tests/easy_video2.py...   PASSED
Running /home/zanza/picamera2/tests/encoder_start_stop.py...    PASSED
Running /home/zanza/picamera2/tests/ffmpeg_abort.py...  PASSED
Running /home/zanza/picamera2/tests/large_datagram.py...    PASSED
Running /home/zanza/picamera2/tests/mjpeg_server.py...  PASSED
Running /home/zanza/picamera2/tests/no_raw.py...    PASSED
Running /home/zanza/picamera2/tests/null_encoder.py...  PASSED
Running /home/zanza/picamera2/tests/mode_test.py...     PASSED
Running /home/zanza/picamera2/tests/multicamera.py...   SKIPPED
Running /home/zanza/picamera2/tests/multicamera_2.py...     SKIPPED
Running /home/zanza/picamera2/tests/preview_cycle_test.py...    PASSED
Running /home/zanza/picamera2/tests/preview_location_test.py...     PASSED
Running /home/zanza/picamera2/tests/preview_start_stop.py...    PASSED
Running /home/zanza/picamera2/tests/quality_check.py...     PASSED
Running /home/zanza/picamera2/tests/qt_gl_preview_test.py...    PASSED
Running /home/zanza/picamera2/tests/stop_slow_framerate.py...   PASSED
Running /home/zanza/picamera2/tests/allocator_test.py...    PASSED
Running /home/zanza/picamera2/tests/allocator_leak_test.py...   PASSED

*** 3 TESTS FAILED! ***
davidplowman commented 8 months ago

Hi again, thanks for the updates. I'm very nearly good to merge - just two final things!

  1. Could you squash your commits together into one? I think it would be tidier and it would be easier for folks to understand if there's just a single commit which has everything correct.
  2. I couldn't get OpenCV to build using pip3 install opencv-python=4.4.0.46. I tried a completely clean up-to-date image with all updates applied, but it failed saying that it couldn't build numpy. Instead if I used sudo apt install python3-opencv before making the venv, it was fine. Should we amend the instructions?

Thanks!

Zanz2 commented 8 months ago

Hi again, thanks for the updates. I'm very nearly good to merge - just two final things!

  1. Could you squash your commits together into one? I think it would be tidier and it would be easier for folks to understand if there's just a single commit which has everything correct.
  2. I couldn't get OpenCV to build using pip3 install opencv-python=4.4.0.46. I tried a completely clean up-to-date image with all updates applied, but it failed saying that it couldn't build numpy. Instead if I used sudo apt install python3-opencv before making the venv, it was fine. Should we amend the instructions?

Thanks!

I've done both of those, as for the requirements, i think i just had that text in the file from before, theres no reason why the version would be pinned to that and why PIL was needed, so i removed the version requirement and set it to be headless (since imshow is never used, and picamera2's gui is used for previewing). Also added instructions for how to install on system python.

Thanks a lot for taking the time to review and for your valuable feedback, working on this was a treat!

davidplowman commented 8 months ago

Thank you very much!!