search

raspberrypi / firmware

This repository contains pre-compiled binaries of the current Raspberry Pi kernel and modules, userspace libraries, and bootloader/GPU firmware.
5.15k stars 1.68k forks source link

Raspberry Pi Camera V2 Color wrong #1361

Open tdub415 opened 4 years ago

tdub415 commented 4 years ago

Describe the bug Using a fresh install of Raspbian (Buster) and Octoprint. When attaching my raspberry pi camera v2 to either a Pi3+ or Pi4, the color balance seems to be constantly auto-balancing with extremely red results. (Detailed here: https://community.octoprint.org/t/raspberry-pi-cam-color/17717/14)

To reproduce Flash Octopi (or manually Built octoprint on a fresh install of Raspbian (Buster)). Configure pi-camera settings.

Expected behaviour Camera color balance to be consistent with how it behaved in Raspbian (Stretch) build of Octoprint I have running on my Pi3+

Actual behaviour Camera color is far too red, and constantly adjusts the balance significantly depending on how close the objects are to the camera

System Copy and paste the results of the raspinfo command in to this section. Alternatively, copy and paste a pastebin link, or add answers to the following questions: pi@raspberrypi:~ $ raspinfo -bash: raspinfo: command not found

davidplowman commented 4 years ago

I'd be interested to see this. Are you able to capture a still image of the problem with the raw data attached? Use something like "raspistill -t 5000 -r -o jpegplusraw.jpg". Also, can you describe roughly what the illumination is. Is it indoors with artificial lighting (what kind)? But also with windows and some daylight entering too? Thanks.

tdub415 commented 4 years ago

`pi@raspberrypi:~ $ raspistill -t 5000 -r -o jpegplusraw.jpg mmal: mmal_vc_component_enable: failed to enable component: ENOSPC mmal: camera component couldn't be enabled mmal: main: Failed to create camera component mmal: Failed to run camera app. Please check for firmware updates

pi@raspberrypi:~ $ `

raspistill doesn't seem to work in buster

Illumination is fluorescent tubes in the main room (somewhat blocked by the enclosure) and a LED array shining down on the build plate (IKEA DIODER strips)

davidplowman commented 4 years ago

Hi, thanks for the reply.

So the illumination sounds interesting, LED lights have been known to cause trouble because of the non-natural spectrum, so it's not implausible that it might confuse things - so it would indeed be good to try and get a raw capture. (You don't by any chance have a means of measuring the colour temperature?)

As regards raspistill not working in Buster - I believe it does work for most people so perhaps we should go through the usual check-list. (Apologies if this is like teaching granny to suck eggs...)

Can't think of anything else for the minute... thanks for your help and please let me know how you get on.

6by9 commented 4 years ago

Did you stop Octopi using the camera before trying raspistill? You can only have one client to the camera at a time. raspistill works fine on Buster.

tdub415 commented 4 years ago

I don't have a way to measure the color temp of the LEDs, but until the buster build for the Pi4, they did not cause an issue with stretch build on the Pi3+. The buster build also produces the issue on the Pi3+.

No worries about going through a basic checklist, it's gonna be the best way to debug the issue. I'm currently using a plugin to manually set the R/B balance, but even with it (which is supposed to set them at a fixed value) I can see some very minor fluctuations like it's starting to try to auto-adjust, but then gets set back to the values I have set in the plugin.

The camera is enabled (had to enable it to get it to be usable in octoprint).

pi@raspberrypi:~ $ sudo vcgencmd get_camera supported=1 detected=1 pi@raspberrypi:~ $

I have tried a few different cables, and ensured that they were all properly seated.

I am 100% certain it is an official module

tdub415 commented 4 years ago

Did you stop Octopi using the camera before trying raspistill? You can only have one client to the camera at a time. raspistill works fine on Buster.

Ah ha! no i didn't, let me find the command to stop octorpint using the camera

pixelfish22 commented 3 years ago

I'm also experiencing this. Was there any resolution or work-around available?

tdub415 commented 3 years ago

I'm also experiencing this. Was there any resolution or work-around available?

I never found a good solution, the only answer I could find was the hardware encoder changed with the pi4 or something, causing the difference in behaviour of the auto white balance.

A workaround I am using was to install the octolapse plugin and use it purely to manually set the color balance. I can still see it trying to auto adjust if I watch closely, but it quickly snaps back to the values I have it forced to.

davidplowman commented 3 years ago

Hi again, as I mentioned previously, I'd like to have a look at this. If someone can post some jpeg+raw captures that exhibit the problem, that would be very helpful. It would also be interesting to know the colour gains that you set manually. Thanks!

tdub415 commented 3 years ago

The octolapse camera settings that work for me (after manually dialing it it) are: Brightness 50, Contrast 10, Saturation 0, Red balance 1000, blue balance 2400, power line frequency 50 hz, sharpness 0, color effect: none, color effect cbcr 32896.

I thought I had replied with the raw images previously but I guess i never had (they were too big to upload, so i guess I forgot to upload them to imgur). Here's a couple of older ones, one I too just a bit ago that do not have the color correction, and one with my color correction settings. https://imgur.com/a/TX2EKuP

Hi again, as I mentioned previously, I'd like to have a look at this. If someone can post some jpeg+raw captures that exhibit the problem, that would be very helpful. It would also be interesting to know the colour gains that you set manually. Thanks!

davidplowman commented 3 years ago

Unfortunately there doesn't seem to be any raw data in those jpegs, they would have to be captured with "raspistill -r".

Nonetheless the red/blue balance numbers are quite interesting. Though I'm not familiar with "octolapse", I assume this means you're specifying a red gain of 1.0 and a blue gain of 2.4.

In the camera tuning for the Pi v2 camera we have the following given as the canonical gains for different colour temperatures:

2498K - red_gain = 1.193 blue_gain = 3.040 2811K - red_gain = 1.353 blue_gain = 2.445 3627K - red_gain = 1.408 blue_gain = 2.101

So the blue balance that you give would seem to suggest a colour temperature around 2800K, but the red balance is completely off the charts. Basically the illumination here is massively redder than would be found on Planckian curve that the AWB is tuned to expect.

It will probably have "worked" on older software because that would "fail over" silently to a "grey world" AWB method. That code won't run any more and the replacement doesn't behave the same. You have to choose your poison, really - if you fail over silently like that then there are cases when a genuine pink object would suddenly go grey. Not sure if there's a solution but I'll give it another ponder.

In the meantime, the realistic workarounds you have are probably:

tdub415 commented 3 years ago

I'm not sure why the raw data isn't there, I used the command you gave me earlier this year "raspistill -t 5000 -r -o jpegplusraw.jpg" for both the old images and the new.

I have experimented with several different light sources (LED, Flourescent, incandescent, and natural sunlight) and all seem to have the same overly red saturated issue.

I'm not sure what the values of the sliders in octolapse translate to, but the sliders range from 1 to 7999.

I'm also not sure what exactly it is that changed with the formula/calculation of awb/color balance between jessie and buster for the camera port, but the overly red balance issue still only happens on both pi3+ on buster and pi 4 (which to my knowledge doesn't have a jessie build). If i revert to an jessie based image on my pi3+, the picture is correct (but there are other issues with the processor throtteling and stuttering under load that i really need to use the 4 for my setup). I'm fine with my manual workaround, the only thing really keeping me coming back to this issue is when I randomly get an email becasue someone else has the issue and asks a question. Trying to keep as much information out there as I can so that hopefully a solution can be found for anyone who needs it.

6by9 commented 3 years ago

Use a file hosting service (eg DropBox or Google Drive) rather than image hosting. Imgur is probably rewriting the file.

janrutger commented 2 years ago

I have the same problem. Pi4B with a new Picam V2. The image is much too red almost purple.

I wonder if it might be a defective chip on the camera because otherwise a lot more people would complain about this. Even with an AWB=grayworld it is anything but color real

UltimateCodeWarrior commented 1 month ago

I have the same problem. Pi4B with a new Picam V2. The image is much too red almost purple.

I wonder if it might be a defective chip on the camera because otherwise a lot more people would complain about this. Even with an AWB=grayworld it is anything but color real

Having the same issue with picam2, bookworm, Pi4B -- image is fuscia 

pi_config = picam.create_video_configuration(main={"size": (x_res, y_res), "format": "RGB888"})
picam.configure(pi_config)

motion_detect_27 2_ydxSYh_ATtqcMFgd4mrNUQ

davidplowman commented 1 month ago

Are you using "noir" cameras? Or are you using some slightly unusual kind of lighting (such as custom LEDs)? It might also help to see what's going on if you capture a DNG file, and then post the output of exiftool when run on that file. Thanks.

UltimateCodeWarrior commented 1 month ago

Camera attaches to the PI via Ribbon Cable. Has Raspberry Pi v2.1 on the board. Sony Exmor Sensor 8MP Still Got it setup for 800x600 right now. No custom LED's, white light LED bulbs in my office. Video has the same tint as still images no matter what the format it's saved as. Preview window shows the same.

Never heard of a DNG file. Or the ExifTool, but I decided to turn it into a research project as none of that stuff installed cleanly or seemed to produced anything tangible to diagnosing the issue with the color tables.

ExifTool Version Number : 12.92 File Name : my_image.dng Directory : /home/minion/CAMBOT File Size : 32 MB File Modification Date/Time : 2024:08:14 02:59:23-07:00 File Access Date/Time : 2024:08:14 02:33:14-07:00 File Inode Change Date/Time : 2024:08:14 02:59:23-07:00 File Permissions : -rw-r--r-- File Type : DNG File Type Extension : dng MIME Type : image/x-adobe-dng Exif Byte Order : Little-endian (Intel, II) Image Width : 6560 Image Height : 2464 Bits Per Sample : 16 Compression : Uncompressed Photometric Interpretation : BlackIsZero Strip Offsets : (Binary data 5326 bytes, use -b option to extract) Rows Per Strip : 4 Strip Byte Counts : (Binary data 3695 bytes, use -b option to extract) Planar Configuration : Chunky Image Size : 6560x2464 Megapixels : 16.2

----Python Script: from ChatGPT that worked in creating a READABLE DNG file

from picamera2 import Picamera2 from PIL import Image import piexif import struct

def save_frame_to_dng(filename="output.dng"):

Initialize the PiCamera2

picam2 = Picamera2()

# Configure the camera for raw capture
raw_config = picam2.create_still_configuration(raw={'format': 'SRGGB10'} )
picam2.configure(raw_config)

# Start the camera
picam2.start()

# Capture a raw frame
raw_frame = picam2.capture_array("raw")

# Stop the camera
picam2.stop()

# Convert raw_frame to 16-bit format (scale from 10-bit to 16-bit)
width, height = raw_frame.shape[1], raw_frame.shape[0]
raw_frame_16bit = bytearray()
for row in raw_frame:
    for pixel in row:
        # Scale the 10-bit pixel value to 16-bit by left-shifting by 6 bits
        pixel_16bit = pixel << 6
        # Pack as 2-byte little-endian format
        raw_frame_16bit.extend(struct.pack('<H', pixel_16bit))

# Convert raw data to a PIL Image
image = Image.frombytes('I;16', (width, height), bytes(raw_frame_16bit))

# Save as TIFF (which is very similar to DNG, though not exactly the same)
image.save("temp_image.tiff", format="TIFF", compression="none")

# Read the TIFF data back
with open("temp_image.tiff", "rb") as tiff_file:
    tiff_data = tiff_file.read()

# Prepare the metadata for DNG
exif_data = {
    "0th": {
        piexif.ImageIFD.Make: "Raspberry Pi".encode(),
        piexif.ImageIFD.Model: "PiCamera2".encode(),
        piexif.ImageIFD.Software: "PiCamera2 DNG Writer".encode(),
    },
    "Exif": {
        piexif.ExifIFD.ISOSpeedRatings: 100,
        piexif.ExifIFD.ExposureTime: (1, 100),
        piexif.ExifIFD.ShutterSpeedValue: (1, 100),
        piexif.ExifIFD.ApertureValue: (2, 1),
        piexif.ExifIFD.BrightnessValue: (1, 1),
    },
    "1st": {}
}

# Serialize the EXIF data
exif_bytes = piexif.dump(exif_data)

# Create the DNG file
with open(filename, 'wb') as f:
    # Write the TIFF data as the base for the DNG
    f.write(tiff_data)

    # Write the EXIF data at the end of the file
    f.write(exif_bytes)

print(f"Frame saved as {filename}")

Example usage

save_frame_to_dng("my_image.dng")

davidplowman commented 1 month ago

ChatGPT is not helping you here. Try something more like this:

import time
from picamera2 import Picamera2

picam2 = Picamera2()
config = picam2.create_still_configuration(raw={'format': 'SRGGB10'} )
picam2.configure(config)
picam2.start()

time.sleep(1)
picam2.capture_file("test.dng", name='raw')

Also, you didn't say if this is a "noir" module. Can you confirm that?

UltimateCodeWarrior commented 1 month ago

Thanks for the Code Snippet, Worked Great :)

ExifTool Version Number : 12.92 File Name : test.dng Directory : /home/minion/CAMBOT File Size : 10 MB File Modification Date/Time : 2024:08:14 03:26:01-07:00 File Access Date/Time : 2024:08:14 03:26:01-07:00 File Inode Change Date/Time : 2024:08:14 03:26:01-07:00 File Permissions : -rw-r--r-- File Type : DNG File Type Extension : dng MIME Type : image/x-adobe-dng Exif Byte Order : Little-endian (Intel, II) Subfile Type : Full-resolution image Image Width : 3280 Image Height : 2464 Bits Per Sample : 10 Compression : Uncompressed Photometric Interpretation : Color Filter Array Make : RaspberryPi Camera Model Name : PiDNG / PiCamera2 Orientation : Horizontal (normal) Samples Per Pixel : 1 Software : PiDNG Tile Width : 3280 Tile Length : 2464 Tile Offsets : 760 Tile Byte Counts : 10102400 CFA Repeat Pattern Dim : 2 2 CFA Pattern 2 : 2 1 1 0 Exposure Time : 1/16 ISO : 558 DNG Version : 1.4.0.0 DNG Backward Version : 1.0.0.0 Black Level Repeat Dim : 2 2 Black Level : 64 64 64 64 White Level : 1023 Color Matrix 1 : 1.4379 -0.4618 -0.0377 0.2628 0.5643 0.1707 0.1437 0.0601 0.2188 Camera Calibration 1 : 1 0 0 0 1 0 0 0 1 Camera Calibration 2 : 1 0 0 0 1 0 0 0 1 As Shot Neutral : 0.8639308855 1 0.4350474202 Baseline Exposure : 1 Calibration Illuminant 1 : D65 Raw Data Unique ID : 3936383337373632313632303030 Profile Name : PiDNG / PiCamera2 Profile Profile Embed Policy : No Restrictions CFA Pattern : [Blue,Green][Green,Red] Image Size : 3280x2464 Megapixels : 8.1 Shutter Speed : 1/16

As far as what NoIR was, I thought that was some sort of Branding.
This Thread Shed some light on what that was all about, basically if it didn't have an IR filter, the captures would be pink: https://forums.r rpi_cam rpi_cam aspberrypi.com/viewtopic.php?t=130443

davidplowman commented 1 month ago

So are those pictures of the camera module that you have? I think the black colour means it's a "noir" sensor. This would mean you need to use the "imx219_noir.json" tuning file. Some of that documentation should explain how to do that.

UltimateCodeWarrior commented 3 weeks ago

Thanks David, that got me pointed in the right direction:

sudo find . -name "imx219*"

./usr/share/libcamera/ipa/rpi/pisp/imx219.json ./usr/share/libcamera/ipa/rpi/pisp/imx219_noir.json ./usr/share/libcamera/ipa/rpi/vc4/imx219.json ./usr/share/libcamera/ipa/rpi/vc4/imx219_noir.json ./usr/share/libcamera/ipa/rkisp1/imx219.yaml

python code: tuning = Picamera2.load_tuning_file("imx219_noir.json") picam = Picamera2(tuning=tuning)

(This will make it nice if I want to switch the camera programmatically from night to day mode and back again.

current_image

davidplowman commented 3 weeks ago

Great, glad it's working better!

UltimateCodeWarrior commented 3 weeks ago

Thanks, I've got a solution cobbled together that will record video and audio and at the time of motion (MSE numpy calc threshold) and write out the .h264 video file from the circular buffer, write out the mono CD quality audio from the microphone, and then use ffmpeg to stitch them into a MP4 for easy playback in browser window. I do write out a current image from the camera and through a polling scheme, I do display it in browser window for monitoring but it's only like 1 frame per 3 seconds. The Ropey Camera project on Github has a much better video preview of the video in the browser window, so I might incorporate some of that work later on. He setup a streaming server. He had some overlays for date / time marking on the video too which was nice. Microphone monitoring in the browser window will be a bit more tricky, right now I have 3-second clips that are played back in a through the Javascript Audio interface. It's laggy, not ideal, but it's something. The FFMPEG transcoding/mixing library is rather slow, but it does work. Might have to offload this work to a powerhouse NAS rPI5 or something better so as to not bog down my capture board. Wonder if it's easier to mix the .h264 and wav file first before transcoding it into the mp4. I'll have to play around with it.