BabitMF / bmf

Cross-platform, customizable multimedia/video processing framework. With strong GPU acceleration, heterogeneous design, multi-language support, easy to use, multi-framework compatible and high performance, the framework is ideal for transcoding, AI inference, algorithm integration, live video streaming, and more.
https://babitmf.github.io/
Apache License 2.0
732 stars 60 forks source link

RuntimeError: [json.exception.type_error.302] type must be string, but is array #69

Closed liujiachang closed 8 months ago

liujiachang commented 8 months ago
pkts = (
    bmf.graph().decode({
        'input_path': stream,
        "loglevel": "quiet",
    })['video']
    .start()  # this will return a packet generator
)

for i, pkt in enumerate(pkts):
    # convert frame to a nd array
    if pkt.is_(bmf.VideoFrame):
        vf = pkt.get(bmf.VideoFrame)
        rgb = mp.PixelInfo(mp.kPF_RGB24)
        np_vf = vf.reformat(rgb).frame().plane(0).numpy()
        # we can add some more processing here, e.g. predicting
        print("frame", i, "shape", np_vf.shape)
    else:
        break

When I used the above code to read the stream, an error occurred. When I switched the video stream to a local video, the error disappeared. I don't know where the problem is, but my video stream is correct. I can use ffmpeg to read the stream and save it as mp4. The following is the error message: image

HuHeng commented 8 months ago

Is your video a live stream? What is the format of the stream url?

liujiachang commented 8 months ago
nohup ffmpeg -re -stream_loop -1 -i 00-10-00.mp4  -c:a aac -strict -2 -ar 44100 -ab 48k  -c:v libx264 -b:v 2000k -vf scale=1920:1080 -f rtsp -rtsp_transport tcp rtsp://*****:1508/ai/stream_1k >> nohup.out 2>&1 &

I push local videos to network streams through ffmpeg. stream: rtsp://*****:1508/ai/stream_1k

sfeiwong commented 8 months ago
pkts = (
    bmf.graph().decode({
        'input_path': stream,
        "loglevel": "quiet",
    })['video']
    .start()  # this will return a packet generator
)

for i, pkt in enumerate(pkts):
    # convert frame to a nd array
    if pkt.is_(bmf.VideoFrame):
        vf = pkt.get(bmf.VideoFrame)
        rgb = mp.PixelInfo(mp.kPF_RGB24)
        np_vf = vf.reformat(rgb).frame().plane(0).numpy()
        # we can add some more processing here, e.g. predicting
        print("frame", i, "shape", np_vf.shape)
    else:
        break

When I used the above code to read the stream, an error occurred. When I switched the video stream to a local video, the error disappeared. I don't know where the problem is, but my video stream is correct. I can use ffmpeg to read the stream and save it as mp4. The following is the error message: image

pls double check the "stream" object is a string but not array in decode config when video stream condition

liujiachang commented 8 months ago

I am certain that the stream object is a string. image

liujiachang commented 8 months ago

After restarting the container, this error disappeared. I think some services may not have started properly. My other question is that when I want to terminate a task while reading the stream, ctrl+c will cause a "Segmentation fault (core dumped)" when terminating the task. What other methods are there to stop the task? The generated 'core. ' file is too large. image

sfeiwong commented 8 months ago

that's a good question since the force quit without segfault or block for a while should be a feature need to be optmized

liujiachang commented 8 months ago

Looking forward to future updates