waveform80 / picamera

A pure Python interface to the Raspberry Pi camera module
https://picamera.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
1.57k stars 355 forks source link

Random motion vectors at the bottom and motion vectors shifted up #650

Closed Jean-Machin closed 4 years ago

Jean-Machin commented 4 years ago

I capture motion vectors on a V1 camera on raspberry 3 B+ using Python 3.7.3:

with picamera.PiCamera( sensor_mode = 2, resolution = (2592//2, 1944//2), framerate = 12) as camera:
    camera.start_recording( 'test.h264', format='h264', motion_output='test.data' )
    try:
        camera.wait_recording(30)
    finally:
        camera.stop_recording()

Then I annotated the frames extracted (ffmpeg) from the h264 with the vectors loaded from the test.data file:

motion_data = np.fromfile(
    'test.data', dtype=[
        ('x', 'i1'),
        ('y', 'i1'),
        ('sad', 'u2'),
        ])

cols = (2592//2 + 15) // 16
cols += 1 
rows = (1944//2 + 15) // 16
frame_count = motion_data.shape[0] // (cols * rows)
motion_data = motion_data.reshape((frame_count, rows, cols))
index=0
for f in motion_data:
    index +=1 
    img = Image.open( 'frame{0:03d}.jpg'.format(index) ) 
    draw = ImageDraw.Draw(img)
    for r in range(rows):
        for c in range(cols):
            x, y, sad = f[r,c]
            if x or y:
                cx = c*16+8
                cy = r*16+8
                draw.line( [ cx, cy, cx+x, cy+y ], fill='#ff0000' )
                draw.ellipse( [ cx-1, cy-1, cx+1, cy+1 ], fill='#ffffff' )
     img.save( 'annot{0:03d}.jpg'.format(index) )

I see two issues: 1) all the frames have a bunch of motion vectors at the bottom where there is no motion: bottom-noise 2) on real motion, the motion vectors seem shifted up by 3 or 4 macro blocks (which about the height of the MV noise at the bottom). motion

I verified that motion_data.shape[0] % (cols * rows) is indeed 0 before the reshape(). The motion shift does not seem a timelag because on horizontal motion it is just shifted up (not left or right).

Now to be transparent, I am a beginner with Linux, the Rasberry, PiCamera, Numpy, image processing, Python... and github.

Jean-Machin commented 4 years ago

I changed the camera and the pi: i cannot reproduce.