raspberrypi / picamera2

New libcamera based python library
BSD 2-Clause "Simplified" License
842 stars 179 forks source link

[BUG]? Network Connections Force Data Into CircularOutput From Undefined Sources? #872

Open ExperimentOwl opened 10 months ago

ExperimentOwl commented 10 months ago

I am reporting only one bug per this issue. This is a weird one that is both intermittent (but easily reproduced within the scope of a day waiting for a bit) and also one that relates to cross-program memory boundaries. I did not see a security policy in place, so if this is a vulnerability, I apologize for the broadcast.

Overview After a recent update (yesterday to bookworm latest) I noticed that sometimes when I initiate a network connection (eg from "curl google.com" or sometimes, though less often, "ping google.com") from most any source, the camera code in https://github.com/raspberrypi/picamera2/blob/main/examples/capture_circular.py seems to change its video stream while waiting for motion detection, as if (title) buffer data is being read from sources other than the camera itself. I am very puzzled.

To Reproduce The script is the link above, I'll copy it here per your instructions:

import time

import numpy as np

from picamera2 import Picamera2
from picamera2.encoders import H264Encoder
from picamera2.outputs import CircularOutput

lsize = (320, 240)
picam2 = Picamera2()
video_config = picam2.create_video_configuration(main={"size": (1280, 720), "format": "RGB888"}, lores={
                                                 "size": lsize, "format": "YUV420"})
picam2.configure(video_config)
picam2.start_preview()
encoder = H264Encoder(1000000, repeat=True)
encoder.output = CircularOutput()
picam2.start()
picam2.start_encoder(encoder)

w, h = lsize
prev = None
encoding = False
ltime = 0

while True:
    cur = picam2.capture_buffer("lores")
    cur = cur[:w * h].reshape(h, w)
    if prev is not None:
        # Measure pixels differences between current and
        # previous frame
        mse = np.square(np.subtract(cur, prev)).mean()
        if mse > 7:
            if not encoding:
                epoch = int(time.time())
                encoder.output.fileoutput = f"{epoch}.h264"
                encoder.output.start()
                encoding = True
                print("New Motion", mse)
            ltime = time.time()
        else:
            if encoding and time.time() - ltime > 5.0:
                encoder.output.stop()
                encoding = False
    prev = cur

picam2.stop_encoder()

Expected behaviour The camera to read from only the camera.

Console Output, Screenshots The system is a Lite Raspberry Pi OS 64 bit build so I don't immediately have screenshots. The output is just that the above loop triggers (falls into a True case) when there has been no change to the camera itself. I think it is happening on the numpy line where we determine change detection for motion (I would love any insights on that equation, which I am far too monke to understand right out).

Hardware : RPi3B+

Additional context The system is extremely minimal, even TriggerHappy and NetworkManager are not enabled in systemd. I have a basic script that connects to the internet manually, and I do all of my work on the cli. Please ask any other questions - I'd like to be helpful, but this is definitely an odd one. Hopefully we can solve it together.

davidplowman commented 10 months ago

Hi, this is very hard to understand. From what you're saying, I take it that activity completely unrelated to the camera, or to the environment the camera is in, can cause the camera images to change? I'm struggling to imagine how that could happen.

Would you be able to capture the pair of images (cur and prev) that the code seems to think has changed? All it does is calculate the mean-square-error between them. It's quite possible that the hard-coded value 7 in there is just way too sensitive, and the loop is just triggering quasi-randomly.

ExperimentOwl commented 8 months ago

Hi, this is very hard to understand. From what you're saying, I take it that activity completely unrelated to the camera, or to the environment the camera is in, can cause the camera images to change? I'm struggling to imagine how that could happen.

Would you be able to capture the pair of images (cur and prev) that the code seems to think has changed? All it does is calculate the mean-square-error between them. It's quite possible that the hard-coded value 7 in there is just way too sensitive, and the loop is just triggering quasi-randomly.

Hi David,

First of all thank you for taking the time to write back to me, and so promptly. I'm sorry I haven't responded sooner, but this project was dropped and the end of the year became suddenly full of emergencies. I'm sure it's probably happened to you at some point too, but I wanted to say that first.

This problem looks to have been a silly cable management issue. Once I opened up the case, it turned out that the edge of the Pi Camera's ribbon cable had been lodged in a high-pressure heat sink, partially stripping the wire, but only over time so I wasn't thinking about it this many months later. It was causing enough feedback to make me think that the above "magic signal" might have been due to the way the code was written, rather than just due to a simple hardware failure. I do feel good about that trailing question mark!

I hope you will forgive that I will have a hard time of reproducing this meaningfully to reply appropriately to your comment, but I think with the problem solved it might be an easy thing to put to rest? But if there are any remaining questions on your mind, just write me back here and I'll try to respond with the urgency of a comatose sloth once more.

Best wishes for all that you celebrate, and a Merry Christmas and happy New Year to you and yours.

davidplowman commented 8 months ago

Hi again, thanks for the update, always nice to know that folks have managed to get things working better. And a Happy New Year to you as well!