search

raspberrypi / picamera2

New libcamera based python library
BSD 2-Clause "Simplified" License
894 stars 189 forks source link

[BUG] libav_h264 encoder uses pyav function wrong #1080

Closed vadlemar closed 3 months ago

vadlemar commented 3 months ago

Describe the bug picamera2/encoders/libav_h264_encoder.py:119 "frame = av.VideoFrame.from_ndarray(m.array, format=self._av_input_format, width=self.width)" results in error "File "av/video/frame.pyx", line 459, in av.video.frame.VideoFrame.from_ndarray TypeError: from_ndarray() got an unexpected keyword argument 'width'"

Checked PyAv github and versions of the av/video/frame.pyx, and it seems like the library never accepted this function signature (=with width argument)

To Reproduce Python 3.12.4 Picamera 0.3.18 PyAv 12.3.0

Code example from the Picamera2 book: ''' import time

from picamera2 import Picamera2 from picamera2.encoders import H264Encoder from picamera2.outputs import FfmpegOutput

picam2 = Picamera2() video_config = picam2.create_video_configuration() picam2.configure(video_config)

encoder = H264Encoder(10000000) output = FfmpegOutput('test.mp4', audio=True)

picam2.start_recording(encoder, output) time.sleep(10) picam2.stop_recording() '''

Expected behaviour Record a short video sample.

Console Output, Screenshots ''' [97:47:56.023152517] [2882699]  INFO Camera camera_manager.cpp:313 libcamera v0.3.0+65-6ddd79b5 [97:47:56.031057428] [2882871]  INFO RPI pisp.cpp:695 libpisp version v1.0.6 b567f0455680 17-06-2024 (10:20:00) [97:47:56.042723424] [2882871]  INFO RPI pisp.cpp:1154 Registered camera /base/axi/pcie@120000/rp1/i2c@80000/imx708@1a to CFE device /dev/media0 and ISP device /dev/media1 using PiSP variant BCM2712_C0 [97:47:56.045110957] [2882699]  INFO Camera camera_manager.cpp:313 libcamera v0.3.0+65-6ddd79b5 [97:47:56.052952238] [2882874]  INFO RPI pisp.cpp:695 libpisp version v1.0.6 b567f0455680 17-06-2024 (10:20:00) [97:47:56.063610237] [2882874]  INFO RPI pisp.cpp:1154 Registered camera /base/axi/pcie@120000/rp1/i2c@80000/imx708@1a to CFE device /dev/media0 and ISP device /dev/media1 using PiSP variant BCM2712_C0 [97:47:56.067390396] [2882699]  WARN V4L2 v4l2_pixelformat.cpp:344 Unsupported V4L2 pixel format RPBP [97:47:56.068208858] [2882699]  INFO Camera camera.cpp:1183 configuring streams: (0) 1280x720-XBGR8888 (1) 1536x864-BGGR_PISP_COMP1 [97:47:56.068336746] [2882874]  INFO RPI pisp.cpp:1450 Sensor: /base/axi/pcie@120000/rp1/i2c@80000/imx708@1a - Selected sensor format: 1536x864-SBGGR10_1X10 - Selected CFE format: 1536x864-PC1B [in#0 @ 0x555592cea8d0] Unknown input format: 'pulse' Error opening input file default. Error opening input files: Invalid argument Exception during process_requests() Traceback (most recent call last): File "/home/rpi/miniforge3/envs/cam/lib/python3.12/site-packages/picamera2/previews/null_preview.py", line 85, in handle_request picam2.process_requests(self) File "/home/rpi/miniforge3/envs/cam/lib/python3.12/site-packages/picamera2/picamera2.py", line 1267, in process_requests encoder.encode(encoder.name, req) File "/home/rpi/miniforge3/envs/cam/lib/python3.12/site-packages/picamera2/encoders/encoder.py", line 210, in encode self._encode(stream, request) File "/home/rpi/miniforge3/envs/cam/lib/python3.12/site-packages/picamera2/encoders/libav_h264_encoder.py", line 119, in _encode frame = av.VideoFrame.from_ndarray(m.array, format=self._av_input_format, width=self.width) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "av/video/frame.pyx", line 459, in av.video.frame.VideoFrame.from_ndarray TypeError: from_ndarray() got an unexpected keyword argument 'width' Exception in thread Thread-5 (thread_func): Traceback (most recent call last): File "/home/rpi/miniforge3/envs/cam/lib/python3.12/threading.py", line 1073, in _bootstrap_inner self.run() File "/home/rpi/miniforge3/envs/cam/lib/python3.12/site-packages/ipykernel/ipkernel.py", line 766, in run_closure _threading_Thread_run(self) File "/home/rpi/miniforge3/envs/cam/lib/python3.12/threading.py", line 1010, in run self._target(*self._args, **self._kwargs) File "/home/rpi/miniforge3/envs/cam/lib/python3.12/site-packages/picamera2/previews/null_preview.py", line 29, in thread_func callback(picam2) File "/home/rpi/miniforge3/envs/cam/lib/python3.12/site-packages/picamera2/previews/null_preview.py", line 85, in handle_request picam2.process_requests(self) File "/home/rpi/miniforge3/envs/cam/lib/python3.12/site-packages/picamera2/picamera2.py", line 1267, in process_requests encoder.encode(encoder.name, req) File "/home/rpi/miniforge3/envs/cam/lib/python3.12/site-packages/picamera2/encoders/encoder.py", line 210, in encode self._encode(stream, request) File "/home/rpi/miniforge3/envs/cam/lib/python3.12/site-packages/picamera2/encoders/libav_h264_encoder.py", line 119, in _encode frame = av.VideoFrame.from_ndarray(m.array, format=self._av_input_format, width=self.width) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "av/video/frame.pyx", line 459, in av.video.frame.VideoFrame.from_ndarray TypeError: from_ndarray() got an unexpected keyword argument 'width' '''

Hardware : Pi 5 4gb / IMX708 camera

davidplowman commented 3 months ago

Hi, and thanks for the report. I think the problem may be that you're not using the Raspberry Pi OS version of PyAv, which should in fact come pre-installed (and is also available in our apt repository). We had to make some slight extensions to PyAv because it couldn't cope with image buffers that have padding on the ends of each row of pixels. There were some other features that were missing, too, you can check them out here: https://github.com/davidplowman/PyAV. I did at one point make a start on trying to upstream some changes, but haven't had time to pursue that, though in principle I would like to. It would also be good to track mainline more closely, but again, I'm not finding the time, and I'm not at this point aware of any updates that we need.

So I'm sorry if that's slightly bad news, but at this point I don't really have any workarounds for you except to use the Raspberry Pi version. You're of course welcome to make use of the changes we made in that repository that I linked to, if that helps.

vadlemar commented 3 months ago

Thank you for the heads up! I am installing all packages in a conda environment on a custom debian distro, so there's no easy way for me to know what packages have been modified for all the Picamera2 functions to work properly that are distributed via the Raspberry OS. Is there a simple way to install all the packages / a list of modified versions which to download and set up, in order for the Picamera2 to work properly? Thanks for your time!

davidplowman commented 3 months ago

Don't know too much about conda, but with a venv we normally recommend using the --system-site-packages option, which is much the easiest way to get the libcamera python bindings. This will also get you compatible versions of Picamera2, and anything else (such as PyAv). Can something like that work for you?

Obviously we wouldn't change an existing package unless we had too. So far as I recall, I think PyAv is probably the only one in this awkward category. I should probably try again to get stuff upstreamed, though it took me many months to get just one tiny change merged, which rather dampened my enthusiasm.

vadlemar commented 3 months ago

I think venv could work for me too. Thanks for pointing me to a new direction to explore and taking the time to reply. I appreciate it :)

44yu5h commented 3 months ago

It's late but might help someone else:

Uninstalled PyAv from pip: pip uninstall av Installed from apt: apt install python3-av

Worked.

davidplowman commented 3 months ago

Great, thanks for the update!

mikkelam commented 4 weeks ago

Maybe you guys should exclude the av package from install_requires then? Then specify it in the readme that user is required to pre install this package

mikkelam commented 3 weeks ago

Maybe you guys should exclude the av package from install_requires then? Then specify it in the readme that user is required to pre install this package

Would you consider a PR with this change @davidplowman

davidplowman commented 3 weeks ago

Yes, sounds like a reasonable thing to do. The problem is principally caused by folks installing a newer version of PyAV through pip, is that right, or is it that PyAV is missing completely from our images (I'm under the impression that installing picamera2 through apt should do the right thing)? Might be worth handling the errors if it falls over, suggesting how to remedy it. Of course, really I need to get everything properly upstreamed, but I wouldn't advise anyone to hold their breath... :(