[HOW-TO] Live stream camera feed using Flask while simultaneously saving video as h264 or mp4

[KarasuY]

I just got a RPI Zero 2W and it's forcing me to use picamera2 instead of picamera, so I have to redo weeks of work to be compatible with the new version. I'm trying to create a flask script to stream the live picam feed while also having an option to start and stop recording which will save the recorded video stream as an H264 or MP4 (I don't care which one) on the the pi.

I've managed to get the streaming on flask working with picamera2 but not saving the file. The code below is based on the working streaming and example 9.3 in the pdf docs. Though the streaming no longer works but a short video file is saved successfully saved as an mp4.

I've managed to get them working one consecutively, where it blocks the streams for 20 seconds while it records and saves then starts the stream. However, I can't get them to work simultaneously even though the example it's based on seems to indicate that they can run at the same time.

import picamera2 #camera module for RPi camera
from picamera2 import Picamera2
from picamera2.encoders import JpegEncoder, H264Encoder
from picamera2.outputs import FileOutput, FfmpegOutput
import io

import subprocess
from flask import Flask, Response
from flask_restful import Resource, Api, reqparse, abort
import atexit
from datetime import datetime
from threading import Condition
import time 

app = Flask(__name__)
api = Api(app)

class StreamingOutput(io.BufferedIOBase):
    def __init__(self):
        self.frame = None
        self.condition = Condition()

    def write(self, buf):
        with self.condition:
            self.frame = buf
            self.condition.notify_all()

#defines the function that generates our frames
def genFrames():
    with picamera2.Picamera2() as camera:
        camera.configure(camera.create_video_configuration(main={"size": (640, 480)}))
        encoder = JpegEncoder()
        output1 = FfmpegOutput('test2.mp4', audio=False) 
        output3 = StreamingOutput()
        output2 = FileOutput(output3)
        encoder.output = [output1, output2]

        camera.start_encoder(encoder) 
        camera.start() 
        output1.start() 
        time.sleep(20) 
        output1.stop() 
        print('done')
        while True:
            with output3.condition:
                output3.condition.wait()
            frame = output3.frame
            yield (b'--frame\r\n'
                b'Content-Type: image/jpeg\r\n\r\n' + frame + b'\r\n')

#defines the route that will access the video feed and call the feed function
class video_feed(Resource):
    def get(self):
        return Response(genFrames(),
                    mimetype='multipart/x-mixed-replace; boundary=frame')

api.add_resource(video_feed, '/cam')

if __name__ == '__main__':
    app.run(debug = False, host = '0.0.0.0', port=5000)

I've also included my original picamera code for reference as to what I'm trying to do.

import io
import picamera
import subprocess
from flask import Flask, Response
from flask_restful import Resource, Api, reqparse, abort
import atexit
from datetime import datetime

app = Flask(__name__)
api = Api(app)

class Camera:
    def __init__(self):
        self.camera = picamera.PiCamera()
        self.camera.resolution = (640, 480)
        self.recording = False
        atexit.register(self.release_cam)

    def start_recording(self, ID):
        if not self.recording:
            now  = datetime.now()
            timestamp = now.strftime("%Y-%m-%d_%H-%M-%S")
            self.output_filename = f'{ID}_{timestamp}.h264'
            self.camera.start_recording(self.output_filename, format='h264')
            self.recording = True
            print(self.output_filename)
            return True
        else:
            print("Already recording")
            return False

    def release_cam(self):
        if self.recording:
            self.camera.stop_recording()
        self.camera.close()

    def stop_recording(self,stopped):
        if self.recording:
            self.camera.stop_recording()
            self.recording = False

            # Use subprocess to upload the recorded video to S3 using AWS CLI
            subprocess.run(['aws', 's3', 'cp', self.output_filename, f's3://dct-cambucket1/{self.output_filename}'])
            return self.output_filename, True
        else:
            print("No recording in progress")
            return None, False

    def get_frame(self):
        stream = io.BytesIO()
        self.camera.capture(stream, format='jpeg', use_video_port=True)
        stream.seek(0)
        return stream.read()

    def convert_to_mp4(self):
        conv_fn = self.output_filename.replace('.h264', '.mp4')
        subprocess.run(['ffmpeg', '-i', self.output_filename, '-c:v', 'copy', conv_fn])
        subprocess.run(['aws', 's3', 'cp', conv_fn, f's3://dct-cambucket1/{conv_fn}'])
        return conv_fn

camera = Camera()

def generate_frames():
    while True:
        frame = camera.get_frame()
        yield (b'--frame\r\n'
               b'Content-Type: image/jpeg\r\n\r\n' + frame + b'\r\n\r\n')

class VideoFeed(Resource):
    def get(self):
        return Response(generate_frames(), mimetype='multipart/x-mixed-replace; boundary=frame')

class StartRec(Resource):
    def get(self, ID):
        start = camera.start_recording(ID)
        if start:
            return {'status': 'True', 'message': 'Recording Started'}, 201
        else:
            return {'status': 'False', 'message': 'Recording already in progress'}, 201

class StopRec(Resource):
    def get(self, ID):
        filename, stop = camera.stop_recording(True)
        if stop:
            link = "https://dct-cambucket1.s3.amazonaws.com/" + filename
            print("Recording stopped and uploaded to S3.")
            return {'status': 'True', 'message': 'Recording Stopped', 'ID': link}, 201
        else:
            return {'status': 'False', 'message': 'No Recording In Progress', 'ID': "None"}, 201
class convVid(Resource):
    def get(self, ID):
        filename = camera.convert_to_mp4()
        link = "https://dct-cambucket1.s3.amazonaws.com/" + filename
        return {'status': 'True', 'message': 'Video Converted', "ID": link}, 201        

api.add_resource(VideoFeed, '/picam/cam')
api.add_resource(StartRec, '/picam/start_rec/<ID>')
api.add_resource(StopRec, '/picam/stop_rec/<ID>')
api.add_resource(convVid, '/picam/convert_vid/<ID>')

if __name__ == '__main__':
    app.run(debug = False, host = '0.0.0.0', port=5000)

I'd appreciate any help. I haven't found anything online that has worked.

[davidplowman]

Hi, I don't really know anything about Flask so this might not be a great suggestion.

I'd be tempted to add start_recording(file) and stop_recording(file) methods to the StreamingOutput class. All these methods do is set some kind of state in the object to signal that a recording needs to start or stop. Then in the existing write method I'd see if I'm currently recording, in which case I'd just dump the data to the file.

There's probably also a tidier way to do this using multiple outputs, but you'll have to experiment a bit for yourself there. One output would be the StreamingOutput that you have, and the other would be a FileOutput where you can start/stop the recording. Check out section 9.3 of the manual.

[KarasuY]

I've managed to get it to the stage where it can stream and record simultaneously. But once I stop the stream I can't start it again, and the start_recording method is essentially useless since the camera init creates the ffmpegoutput (which creates the file and starts recording). I need to be able to start and stop recording, and set new file names for the new recordings which I can't do. I've included the latest version of the code.

import picamera2 #camera module for RPi camera
from picamera2 import Picamera2
from picamera2.encoders import JpegEncoder, H264Encoder
from picamera2.outputs import FileOutput, FfmpegOutput
import io

import subprocess
from flask import Flask, Response
from flask_restful import Resource, Api, reqparse, abort
import atexit
from datetime import datetime
from threading import Condition
import time 

app = Flask(__name__)
api = Api(app)

class Camera:
    def __init__(self):
        self.camera = picamera2.Picamera2()
        self.camera.configure(self.camera.create_video_configuration(main={"size": (640, 480)}))
        self.encoder = JpegEncoder()
        self.fileOut = FfmpegOutput('test2.mp4', audio=False) #StreamingOutput()
        self.streamOut = StreamingOutput()
        self.streamOut2 = FileOutput(self.streamOut)
        self.encoder.output = [self.fileOut, self.streamOut2]

        self.camera.start_encoder(self.encoder) 
        self.camera.start() 

    def get_frame(self):    
        self.camera.start()
        with self.streamOut.condition:
            self.streamOut.condition.wait()
            self.frame = self.streamOut.frame
        return self.frame

    #doesnt do anything 
    def start_recording(self):
        print('recording started')
        self.fileOut.start()
        return True

    def stop_recording(self):
        print('recording stopped')
        self.fileOut.stop()
        return True

class StreamingOutput(io.BufferedIOBase):
    def __init__(self):
        self.frame = None
        self.condition = Condition()

    def write(self, buf):
        with self.condition:
            self.frame = buf
            self.condition.notify_all()

#defines the function that generates our frames
camera = Camera()

def genFrames():
    while True:
        frame = camera.get_frame()
        yield (b'--frame\r\n'
               b'Content-Type: image/jpeg\r\n\r\n' + frame + b'\r\n\r\n')

#defines the route that will access the video feed and call the feed function
class VideoFeed(Resource):
    def get(self):
        return Response(genFrames(),
                    mimetype='multipart/x-mixed-replace; boundary=frame')

class StartRec(Resource):
    def get(self):
        camera.start_recording()
        return {'Status': 'True'}

class StopRec(Resource):
    def get(self):
        camera.stop_recording()
        return {'Status': 'True'}

api.add_resource(VideoFeed, '/cam')
api.add_resource(StartRec, '/start_rec')
api.add_resource(StopRec, '/stop_rec')
# api.add_resource(convVid, '/picam/convert_vid/<ID>')

if __name__ == '__main__':
    app.run(debug = False, host = '0.0.0.0', port=5000)

[davidplowman]

Hi, glad that you're making progress! I think you probably need to create a new FfmpegOutput instance to start a new recording. I guess you'd have to start it, and then put it in the encoder's list of outputs (replacing the old one). Would that work?

[KarasuY]

I managed to fix the encoding issue by creating my own version of the ffmpegout.py file. I'm adding the solution in here incase anyone else ends up looking for it. Change the video_input and video_codec lines to:

video_input = ['-use_wallclock_as_timestamps', '1', '-thread_queue_size', '64', '-i', '-']
video_codec = ['-c:v', 'libx264', 'preset', 'ultrafast']

This is set to run on a raspberry pi zero.

[mechanburg]

@KarasuY I'm looking to use your code to create the same thing, without saving to video file. I'd like to access the stream from a browser on an external computer, but I cannot access the flask app. What am I missing, how do I view the stream?

[KarasuY]

When you run the code it should output a link including the IP address and port number. To access the stream use /cam to the end of that link. The only issue is if your RPI doesn't have a web browser the link won't work on a seperate computer. You'll need to use a service like Ngrok (they have a free version) to be able to stream it externally. If your pi doesn't have a web browser follow the docs I attached below to install ngrok.
Follow the instructions on the website to set up the configuration. The link at the bottom of the document is an example based on the randomly generated link that I was given on the free version. You will be generated a different one. How to install ngrok on RPI.pdf

[mechanburg]

It's oddly not even getting to the Flask/hosting part of the code, just provides camera status output, without ever starting the server.

[0:00:36.399202940] [6650] INFO Camera camera_manager.cpp:284 libcamera v0.2.0+46-075b54d5 [0:00:36.434560606] [6653] WARN RPiSdn sdn.cpp:39 Using legacy SDN tuning - please consider moving SDN inside rpi.denoise [0:00:36.437240588] [6653] INFO RPI vc4.cpp:447 Registered camera /base/soc/i2c0mux/i2c@0/imx477@1a to Unicam device /dev/media0 and ISP device /dev/media3 [0:00:36.437368421] [6653] INFO RPI pipeline_base.cpp:1144 Using configuration file '/usr/share/libcamera/pipeline/rpi/vc4/rpi_apps.yaml' [0:00:36.441237921] [6650] INFO Camera camera_manager.cpp:284 libcamera v0.2.0+46-075b54d5 [0:00:36.474482606] [6656] WARN RPiSdn sdn.cpp:39 Using legacy SDN tuning - please consider moving SDN inside rpi.denoise [0:00:36.476891440] [6656] INFO RPI vc4.cpp:447 Registered camera /base/soc/i2c0mux/i2c@0/imx477@1a to Unicam device /dev/media0 and ISP device /dev/media3 [0:00:36.476970699] [6656] INFO RPI pipeline_base.cpp:1144 Using configuration file '/usr/share/libcamera/pipeline/rpi/vc4/rpi_apps.yaml' [0:00:36.484344218] [6650] INFO Camera camera.cpp:1183 configuring streams: (0) 1280x720-RGB888 (1) 2028x1080-SBGGR12_CSI2P [0:00:36.484819773] [6656] INFO RPI vc4.cpp:611 Sensor: /base/soc/i2c0mux/i2c@0/imx477@1a - Selected sensor format: 2028x1080-SBGGR12_1X12 - Selected unicam format: 2028x1080-pBCC

[KarasuY]

Have you tested the basic camera access by trying to take a photo or something similar using command line? Though to be honest I gave up on picamera2. I spent days trying to get it to work and eventually gave up. I re-installed a legacy OS and and use picamera.

[mechanburg]

Ah thanks, definitely able to view the camera and got it streaming via http server, thought flask would add some extra power. Thanks anyway.

[allphasepi]

Just been looking at your code and came up with these addition. from picamera2.outputs import CircularOutput

encoder = H264Encoder() output = CircularOutput()

self.camera.start_encoder(self.encoder) self.camera.start_recording(encoder, output)

doesnt do anything

def start_recording(self):
    print('recording started')
    #self.fileOut.start()
    return True

def stop_recording(self):
    print('recording stopped')
    #self.fileOut.stop()
    return True

class StartRec(Resource): def get(self): print("Vid Record") output.fileoutput = "file.h264" output.start() return {'Status': 'True'}

class StopRec(Resource): def get(self): print("Vid Stop") output.stop() return {'Status': 'True'}

I seem to be able to start and stop video recording to file fine.

[allphasepi]

I've posted a fully working example with recoding video, taking a picture and recording from an I2C microphone here https://github.com/allphasepi/Webcam/tree/main

[IcyG1045]

I have forked the repo and added many features such as motion detection and email notifications.

https://github.com/IcyG1045/CM4Cam