search

raspberrypi / picamera2

New libcamera based python library
BSD 2-Clause "Simplified" License
890 stars 188 forks source link

[BUG] capture_metadata output key "PispStatsOutput" #1045

Open mgineer85 opened 5 months ago

mgineer85 commented 5 months ago

Describe the bug printing capture_metadata() to console holds a key PispStatsOutput with many numbers I am not sure what they are for. It seems a bit random though if it is printed or not.

pi@wiggle:~/wiggle/1picamera2-dualstills $ python algo.py 
[1:23:49.528169747] [7512]  INFO Camera camera_manager.cpp:284 libcamera v0.2.0+120-eb00c13d
[1:23:49.539381882] [7515]  INFO RPI pisp.cpp:695 libpisp version v1.0.5 999da5acb4f4 17-04-2024 (14:29:29)
[1:23:49.549379943] [7515]  INFO RPI pisp.cpp:1154 Registered camera /base/axi/pcie@120000/rp1/i2c@88000/imx708@1a to CFE device /dev/media1 and ISP device /dev/media3 using PiSP variant BCM2712_C0
[1:23:49.549502369] [7515]  INFO RPI pisp.cpp:695 libpisp version v1.0.5 999da5acb4f4 17-04-2024 (14:29:29)
[1:23:49.559185521] [7515]  INFO RPI pisp.cpp:1154 Registered camera /base/axi/pcie@120000/rp1/i2c@80000/imx708@1a to CFE device /dev/media2 and ISP device /dev/media4 using PiSP variant BCM2712_C0
[1:23:49.560911633] [7512]  INFO Camera camera_manager.cpp:284 libcamera v0.2.0+120-eb00c13d
[1:23:49.572270842] [7520]  INFO RPI pisp.cpp:695 libpisp version v1.0.5 999da5acb4f4 17-04-2024 (14:29:29)
[1:23:49.582549347] [7520]  INFO RPI pisp.cpp:1154 Registered camera /base/axi/pcie@120000/rp1/i2c@88000/imx708@1a to CFE device /dev/media1 and ISP device /dev/media3 using PiSP variant BCM2712_C0
[1:23:49.582627718] [7520]  INFO RPI pisp.cpp:695 libpisp version v1.0.5 999da5acb4f4 17-04-2024 (14:29:29)
[1:23:49.592498593] [7520]  INFO RPI pisp.cpp:1154 Registered camera /base/axi/pcie@120000/rp1/i2c@80000/imx708@1a to CFE device /dev/media2 and ISP device /dev/media4 using PiSP variant BCM2712_C0
[1:23:49.594067908] [7512]  INFO Camera camera_manager.cpp:284 libcamera v0.2.0+120-eb00c13d
[1:23:49.605284673] [7525]  INFO RPI pisp.cpp:695 libpisp version v1.0.5 999da5acb4f4 17-04-2024 (14:29:29)
[1:23:49.615596418] [7525]  INFO RPI pisp.cpp:1154 Registered camera /base/axi/pcie@120000/rp1/i2c@88000/imx708@1a to CFE device /dev/media1 and ISP device /dev/media3 using PiSP variant BCM2712_C0
[1:23:49.615693159] [7525]  INFO RPI pisp.cpp:695 libpisp version v1.0.5 999da5acb4f4 17-04-2024 (14:29:29)
[1:23:49.625428590] [7525]  INFO RPI pisp.cpp:1154 Registered camera /base/axi/pcie@120000/rp1/i2c@80000/imx708@1a to CFE device /dev/media2 and ISP device /dev/media4 using PiSP variant BCM2712_C0
[1:23:49.627747147] [7512]  WARN V4L2 v4l2_pixelformat.cpp:344 Unsupported V4L2 pixel format RPBP
[1:23:49.628433480] [7512]  INFO Camera camera.cpp:1183 configuring streams: (0) 4608x2592-BGR888 (1) 4608x2592-GRBG_PISP_COMP1
[1:23:49.628554054] [7525]  INFO RPI pisp.cpp:1450 Sensor: /base/axi/pcie@120000/rp1/i2c@88000/imx708@1a - Selected sensor format: 4608x2592-SGRBG10_1X10 - Selected CFE format: 4608x2592-PC1G
[1:23:49.636424984] [7512]  WARN V4L2 v4l2_pixelformat.cpp:344 Unsupported V4L2 pixel format RPBP
[1:23:49.636939818] [7512]  INFO Camera camera.cpp:1183 configuring streams: (0) 4608x2592-BGR888 (1) 4608x2592-GRBG_PISP_COMP1
[1:23:49.637060429] [7525]  INFO RPI pisp.cpp:1450 Sensor: /base/axi/pcie@120000/rp1/i2c@80000/imx708@1a - Selected sensor format: 4608x2592-SGRBG10_1X10 - Selected CFE format: 4608x2592-PC1G
{'SensorTimestamp': 5031973388000, 'ScalerCrop': (0, 0, 4608, 2592), 'AfPauseState': 0, 'ColourCorrectionMatrix': (1.5335766077041626, -0.3524482846260071, -0.18112580478191376, -0.30422690510749817, 1.6739041805267334, -0.36966726183891296, 0.010749365203082561, -0.5653839707374573, 1.5546445846557617), 'PispStatsOutput': (0, 172, 51, 0, 0, 205, 111, 0, 128, 110, 52, 0, 136, 11, 0, 0, 0, 220, 54, 0, 0, 69, 119, 0, 192, 106, 55, 0, 136, 11, 0, 0, 128, 219, 58, 0, 64, 164, 128, 0, 0, 210, 59, 0, 136, 11, 0, 0, 192, 25, 64, 0, 64, 69, 140, 0, 192, 19, 65, 0, 136, 11, 0, 0, 128, 101, 70, 0, 192, 147, 154, 0, 128, 214, 71, 0, 136, 11, 0, 0, 128, 184, 77, 0, 128, 175, 170, 0, 128, 132, 79, 0, 136, 11, 0, 0, 64, 18, 85, 0, 128, 144, 187, 0, 0, 136, 87, 0, 136, 11, 0, 0, 0, 146, 93, 0, 192, 152, 205, 0, 128, 33, 96, 0, 136, 11, 0, 0, 0, 119, 102, 0, 0, 97, 224, 0, 64, 123, 104, 0, 136, 11, 0, 0, 0, 11, 114, 0, 192, 159, 248, 0, 192, 53, 116, 0, 136, 11, 0, 0, 128, 215, 127, 0, 64, 126, 20, 1, 128, 114, 129, 0, 136, 11, 0, 0, 128, 254, 142, 0, 128, 245, 51, 1, 0, 162, 144, 0, 136, 11, 0, 0, 128, 202, 156, 0, 128, 15, 80, 1, 0, 143, 157, 0, 136, 11, 0, 0, 64, 113, 170, 0, 192, 232, 105, 1, 0, 88, 169, 0, 136, 11, 0, 0, 128, 220, 181, 0, 64, 17, 130, 1, 64, 15, 180, 0, 136, 11, 0, 0, 0, 234, 189, 0, 64, 236, 146, 1, 192, 98, 187, 0, 136, 11, 0, 0, 64, 239, 193, 0, 128, 52, 154, 1, 128, 108, 190, 0, 136, 11, 0, 0, 0, 173, 199, 0, 0, 226, 167, 1, 192, 80, 196, 0, 136, 11, 0, 0, 0, 178, 199, 0, 64, 168, 169, 1, 0, 249, 197, 0, 136, 11, 0, 0, 64, 55, 192, 0, 192, 97, 157, 1, 64, 228, 191, 0, 136, 11, 0, 0, 64, 18, 183, 0, 0, 195, 140, 1, 0, 196, 183, 0, 136, 11, 0, 0, 128, 46, 175, 0, 128, 136, 126, 1, 128, 174, 177, 0, 136, 11, 0, 0, 64, 178, 164, 0, 128, 93, 105, 1, 0, 216, 167, 0, 136, 11, 0, 0, 0, 214, 152, 0, 64, 36, 81, 1, 128, 2, 157, 0, 136, 11, 0, 0, 0, 132, 138, 0, 0, 242, 49, 1, 128, 40, 142, 0, 136, 11, 0, 0, 192, 10, 121, 0, 128, 188, 10, 1, 192, 42, 123, 0, 136, 11, 0, 0, 64, 86, 105, 0, 192, 60, 234, 0, 64, 52, 108, 0, 136, 11, 0, 0, 64, 192, 91, 0, 0, 223, 202, 0, 0, 170, 93, 0, 136, 11, 0, 0, 128, 213, 77, 0, 0, 228, 171, 0, 64, 207, 78, 0, 136, 11, 0, 0, 128, 39, 65, 0, 192, 41, 143, 0, 192, 149, 65, 0, 136, 11, 0, 0, 0, 1, 55, 0, 128, 47, 119, 0, 128, 90, 54, 0, 136, 11, 0, 0, 0, 176, 46, 0, 0, 180, 100, 0, 128, 5, 46, 0, 136, 11, 0, 0, 128, 155, 53, 0, 64, 94, 116, 0, 0, 161, 54, 0, 136, 11, 0, 0, 192, 133, 56, 0, 128, 41, 123, 0, 0, 77, 57, 0, 136, 11, 0, 0, 0, 64, 60, 0, 64, 194, 131, 0, 192, 86, 61, 0, 136, 11, 0, 0, 64, 37, 65, 0, 0, 178, 142, 0, 192, 204, 66, 0, 136, 11, 0, 0, 0, 199, 70, 0, 0, 13, 155, 0, 64, 61, 72, 0, 136, 11, 0, 0, 192, 165, 76, 0, 192, 155, 168, 0, 192, 125, 78, 0, 136, 11, 0, 0, 192, 1, 84, 0, 192, 25, 184, 0, 192, 180, 85, 0, 136, 11, 0, 0, 64, 143, 92, 0, 0, 0, 202, 0, 0, 4, 94, 0, 136, 11, 0, 0, 128, 51, 102, 0, 64, 85, 221, 0, 192, 109, 103, 0, 136, 11, 0, 0, 64, 25, 114, 0, 128, 84, 246, 0, 192, 188, 114, 0, 136, 11, 0, 0, 0, 193, 126, 0, 0, 160, 14, 1, 128, 70, 127, 0, 136, 11, 0, 0, 0, 187, 141, 0, 64, 48, 45, 1, 192, 234, 140, 0, 136, 11, 0, 0, 0, 163, 157, 0, 128, 209, 76, 1, 64, 47, 156, 0, 136, 11, 0, 0, 0, 106, 171, 0, 128, 26, 104, 1, 128, 99, 168, 0, 136, 11, 0, 0, 64, 229, 181, 0, 64, 149, 124, 1, 192, 112, 177, 0, 136, 11, 0, 0, 0, 129, 189, 0, 0, 107, 140, 1, 64, 120, 184, 0, 136, 11, 0, 0, 64, 107, 197, 0, 0, 60, 157, 1, 192, 114, 191

...
cut
...

 142, 2, 0, 0, 0, 0, 48, 22, 29, 5, 0, 0, 0, 0, 16, 233, 60, 10, 0, 0, 0, 0, 16, 222, 13, 33, 0, 0, 0, 0, 208, 205, 193, 169, 0, 0, 0, 0, 144, 30, 62, 238, 0, 0, 0, 0, 128, 88, 117, 8, 1, 0, 0, 0, 96, 188, 92, 71, 0, 0, 0, 0, 16, 115, 107, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0), 'FocusFoM': 999, 'ExposureTime': 29997, 'AfState': 0, 'DigitalGain': 1.0001816749572754, 'FrameDuration': 69669, 'SensorTemperature': 50.0, 'AeLocked': True, 'AnalogueGain': 1.6050156354904175, 'LensPosition': 1.0, 'ColourGains': (1.8967149257659912, 1.9803507328033447), 'ColourTemperature': 4385, 'Lux': 296.7579040527344, 'SensorBlackLevels': (4096, 4096, 4096, 4096)}
{'SensorTimestamp': 5032069444000, 'ScalerCrop': (0, 0, 4608, 2592), 'AfPauseState': 0, 'ExposureTime': 29997, 'AfState': 0, 'DigitalGain': 1.0006718635559082, 'FrameDuration': 69669, 'SensorTemperature': 50.0, 'AeLocked': True, 'AnalogueGain': 1.718120813369751, 'LensPosition': 1.0, 'ColourGains': (1.9274909496307373, 1.9470425844192505), 'ColourTemperature': 4476, 'Lux': 277.05743408203125, 'SensorBlackLevels': (4096, 4096, 4096, 4096), 'FocusFoM': 767, 'ColourCorrectionMatrix': (1.5322445631027222, -0.35252487659454346, -0.17971812188625336, -0.29654285311698914, 1.673172116279602, -0.37661927938461304, 0.01271903607994318, -0.5671645402908325, 1.5544555187225342)}

To Reproduce Execute following script on latest RPi5 with two CM3 and system downloaded yesterday

#!/usr/bin/python3

import time

import libcamera
from picamera2 import Picamera2

if len(Picamera2.global_camera_info()) <= 1:
    print("SKIPPED (one camera)")
    quit()

picam2a = Picamera2(0)
config2a = picam2a.create_still_configuration()
config2a["transform"] = libcamera.Transform(hflip=0, vflip=1)
picam2a.configure(config2a)

picam2b = Picamera2(1)
config2b = picam2b.create_still_configuration()
config2b["transform"] = libcamera.Transform(hflip=0, vflip=1)
picam2b.configure(config2b)

picam2a.start()
picam2b.start()

time.sleep(2)
picam2a.capture_metadata()
picam2b.capture_metadata()
print(picam2a.capture_metadata())
print(picam2b.capture_metadata())

picam2a.capture_file("00.jpg")
picam2b.capture_file("01.jpg")

picam2a.stop()
picam2b.stop()

Expected behaviour No mess in the console with useless numbers ;)

Console Output, Screenshots See above.

Hardware : RPi5 with two CM3 and system Bookworm downloaded yesterday.

davidplowman commented 5 months ago

Hi, thanks for the report and sorry for the problems. There does seem to be something slightly wrong underneath in libcamera which we're going to have to look at.

The first problem is that the output of these statistics should be disabled by default, only it isn't. Obviously we'll fix that first, and then at least the obvious symptom that you're seeing will disappear.

The next problem is that the statistics (when enabled...) should appear on every frame. Unfortunately when the sensor is running faster than 30fps then it's failing to do this. We'll need to fix that too.

Finally, you possibly don't want to know what these numbers are all about, but if you did, they're platform specific binary data that comes from the ISP containing the image statistics that the hardware has calculated. The numbers are actually an instance of this structure.