search

chenxinfeng4 / ffmpegcv

The ffmpegcv is a ffmpeg backbone for open-cv like Video Reader and Writer
162 stars 24 forks source link

.isOpened() #18

Closed bertelschmitt closed 1 year ago

bertelschmitt commented 1 year ago

Is there an equivalent of cv2.VideoWriter's .isOpened()?

How to tell that a videowriter file handle is open or not?

chenxinfeng4 commented 1 year ago

The vid writer has 3 status. 

vid = ffmpegcv.VideoWriter(...)
isReady = vid.waitInit;

   # isRunning or isReleased. Try this
    return_code = vid.process.poll()
    if return_code is None:
        print("isRunning ...")
    else:
        print("isReleased")

I'll add opencv like .isOpened() in next release.

bertelschmitt commented 1 year ago

Xie xie.

I need to mirror cv2 functionality. For the time being, I have added:

class FFmpegWriter:
    def __init__(self):
        self.iframe = -1
        self.isOpen = False
    @staticmethod
    def VideoWriter(filename, codec, fps, frameSize, pix_fmt, bitrate=None):
        if codec is None:
            codec = "h264"
        elif not isinstance(codec, str):
            codec = "h264"
            warnings.simplefilter(
                """
                Codec should be a string. Eg `h264`, `h264_nvenc`. 
                You may used CV2.VideoWriter_fourcc, which will be ignored.
                """
            )

        vid = FFmpegWriter()
        vid.fps, vid.size = fps, frameSize
        vid.width, vid.height = vid.size if vid.size else (None, None)
        vid.codec, vid.pix_fmt, vid.filename = codec, pix_fmt, filename
        vid.waitInit = True
        vid.bitrate = bitrate
        vid.isOpen = True
        return vid
    def release(self):
        if hasattr(self, "process"):
            release_process(self.process)
            self.isOpen = False

    def isOpened(self):
        return(self.isOpen)

``

chenxinfeng4 commented 1 year ago

Try this to see if it works for you. I'll release this version after testing in days.

pip install -U git+https://github.com/chenxinfeng4/ffmpegcv
bertelschmitt commented 1 year ago

Will do when back at my desk.

Thank you

B

Get TypeApp for Android

On Aug 8, 2023, 03:04, at 03:04, ChenXF @.***> wrote:

Try this to see if it works for you. I'll release this version after testing in days.

pip install -U git+https://github.com/chenxinfeng4/ffmpegcv

-- Reply to this email directly or view it on GitHub: https://github.com/chenxinfeng4/ffmpegcv/issues/18#issuecomment-1668355600 You are receiving this because you authored the thread.

Message ID: @.***>

chenxinfeng4 commented 1 year ago

The .isOpened() is appended to the VideoReader and the VideoWriter. Already released to the pypi.

bertelschmitt commented 1 year ago

Hi, Chen:

Tested both, and both work.

Also, please be informed that ffmpegcv cv.videowriter solved a cv2 problem I’ve tried tracking down for weeks without any luck. Following a release() of the cv2 videowriter handle, the system would randomly crash, usually without an error message. This left corrupted video files.

After I installed ffmpegcv, the system ran for 24 hours without a hitch, all produced video files are good. I am capturing video from multiple sources, I process them via YOLO, and save them when YOLO has detected an object. All of that in multiple python processes.. The random crashes happened after cv2 upgrades from 4.4. With ffmpegcv, the crashes appear to be solved.

Your videocapture is solid when capturing frames from files. I wasn’t successful capturing rtsp streams. I you want specifics, please let me know. I’m back to cv2.videocapture, but would love to use ffmpegcv, because of its CUDA support.

If you need help with the documentation, please let me know.

chenxinfeng4 commented 1 year ago

the system ran for 24 hours without a hitch, all produced video files are good.

Thanks for your feed back. As you mention cv2. The difference between cv2 and ffmpegcv is that the latter used real binary executable (ffmpeg) as backend, and used the Pipe to transfer the image frames. It's reliable, but sometimes the Pipe caused performance penalty. So the ffmpegcv may not be faster than cv2 even when ffmpegcv using a GPU 😅.

I process them via YOLO

Similar to your case, the ffmpegcv for me is just to fit my CNN video processing. And the CNN input image is usually fixed HW size (640x640 for yolo) and RGB pixel order. The ffmpegcv can set the ROI and pixel order, which will save the CPU time. This feature may help you.

I wasn’t successful capturing rtsp streams.

Did you try the VideoCaptureStream for rtsp streaming? I've passed the test on RTMP/HTTP stream.

bertelschmitt commented 1 year ago

Thanks for the reply.

I’m not looking for speed. I capture and write at 8fps, that’s good enough for me. I’m trying to save power, the mpeg decode takes a lot of power, and I’d like to find out whether CUDA can be more power efficient

Interesting. I’ll get ffmpeg with CUDA going.

Yes, I did. I’ll run some tests over the weekend, and report back.

B

bertelschmitt commented 1 year ago

FYI:

Ran a few VideoCaptureStream tests. This works:

import cv2

@.***:554/stream1"

cap = cv2.VideoCapture(stream_url)

print("got cap")

while True:

ret, frame = cap.read()

if not ret:

    break

print(frame)

cv2.imshow('Test', frame)

if cv2.waitKey(125) == ord('q'):

    break

=================================================================

This dies during "cap = cv2.VideoCapture(stream_url)." "Got cap" is never printed. Frame is never printed, nor shown.

import cv2

import ffmpegcv

@.***:554/stream1"

cap = ffmpegcv.VideoCaptureStream(stream_url, cv2.CAP_FFMPEG)

cap = cv2.VideoCapture(stream_url)

print("got cap")

while True:

ret, frame = cap.read()

if not ret:

    break

print(frame)

cv2.imshow('Test', frame)

if cv2.waitKey(125) == ord('q'):

    break

ctl-c always breaks suchly, probably telling you where it is stuck:

Traceback (most recent call last):

File "./fcvtest.py", line 9, in 

cap = ffmpegcv.VideoCaptureStream(stream_url)

File "/usr/local/bin/.virtualenvs/catdetector/lib/python3.8/site-packages/ffmpegcv/init.py", line 398, in VideoCaptureStream

return FFmpegReaderStream.VideoReader(

File "/usr/local/bin/.virtualenvs/catdetector/lib/python3.8/site-packages/ffmpegcv/ffmpeg_reader_stream.py", line 22, in VideoReader

videoinfo = get_info(stream_url)

File "/usr/local/bin/.virtualenvs/catdetector/lib/python3.8/site-packages/ffmpegcv/stream_info.py", line 8, in get_info

output = subprocess.check_output(cmd, shell=True)

File "/usr/local/bin/lib/python3.8/subprocess.py", line 415, in check_output

return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,

File "/usr/local/bin/lib/python3.8/subprocess.py", line 495, in run

stdout, stderr = process.communicate(input, timeout=timeout)

File "/usr/local/bin/lib/python3.8/subprocess.py", line 1015, in communicate

stdout = self.stdout.read()

KeyboardInterrupt

=========================================================

This fails immediately on cap = ffmpegcv.VideoCaptureStream(stream_url, cv2.CAP_FFMPEG)

import cv2

import ffmpegcv

@.***:554/stream1"

cap = ffmpegcv.VideoCaptureStream(stream_url, cv2.CAP_FFMPEG)

print("got cap")

while True:

ret, frame = cap.read()

if not ret:

    break

print(frame)

cv2.imshow('Test', frame)

if cv2.waitKey(125) == ord('q'):

    break

Traceback (most recent call last):

File "./fcvtest.py", line 9, in 

cap = ffmpegcv.VideoCaptureStream(stream_url, cv2.CAP_FFMPEG)

File "/usr/local/bin/.virtualenvs/catdetector/lib/python3.8/site-packages/ffmpegcv/init.py", line 398, in VideoCaptureStream

return FFmpegReaderStream.VideoReader(

File "/usr/local/bin/.virtualenvs/catdetector/lib/python3.8/site-packages/ffmpegcv/ffmpeg_reader_stream.py", line 20, in VideoReader

assert pix_fmt in ["rgb24", "bgr24", "yuv420p", "nv12", "gray"]

AssertionError

=============================================================================

No big deal, I'm back to using cv2 for capture, your videowriter works flawlessly.

Best, B

From: ChenXF @.> Sent: Wednesday, August 9, 2023 2:10 To: chenxinfeng4/ffmpegcv @.> Cc: bertelschmitt @.>; Author @.> Subject: Re: [chenxinfeng4/ffmpegcv] .isOpened() (Issue #18)

Closed #18 https://github.com/chenxinfeng4/ffmpegcv/issues/18 as completed.

— Reply to this email directly, view it on GitHub https://github.com/chenxinfeng4/ffmpegcv/issues/18#event-10038120946 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AMCPYCVO3WXN3MZTK7VKHKDXUJXHHANCNFSM6AAAAAA3HILYLI . You are receiving this because you authored the thread.Message ID: @.***>

chenxinfeng4 commented 1 year ago

Error: cap = ffmpegcv.VideoCaptureStream(stream_url, cv2.CAP_FFMPEG)

=>

Correct: cap = ffmpegcv.VideoCaptureStream(stream_url)

bertelschmitt commented 1 year ago

I know. I gave you both versions, one with cv2.CAP_FFMPEG, one without.

B

From: ChenXF @.> Sent: Thursday, August 10, 2023 2:55 To: chenxinfeng4/ffmpegcv @.> Cc: bertelschmitt @.>; Author @.> Subject: Re: [chenxinfeng4/ffmpegcv] .isOpened() (Issue #18)

Error: cap = ffmpegcv.VideoCaptureStream(stream_url, cv2.CAP_FFMPEG)

=>

Correct: cap = ffmpegcv.VideoCaptureStream(stream_url)

— Reply to this email directly, view it on GitHub https://github.com/chenxinfeng4/ffmpegcv/issues/18#issuecomment-1671887877 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AMCPYCUI6JYNA66X4IKHQL3XUPFG5ANCNFSM6AAAAAA3HILYLI . You are receiving this because you authored the thread. https://github.com/notifications/beacon/AMCPYCX6J4N5DYBMJ6PRKH3XUPFG5A5CNFSM6AAAAAA3HILYLKWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTTDU36AK.gif Message ID: @. @.> >

chenxinfeng4 commented 1 year ago

Can you run the code below and post the result?

ffprobe -v quiet -print_format xml -select_streams v:0 -show_format -show_streams rtsp://YOURSTREAM

And this.

from ffmpegcv.stream_info import get_info
get_info('rtsp://YOURSTREAM')
bertelschmitt commented 1 year ago

ffprobe -v quiet -print_format xml -select_streams v:0 -show_format -show_streams @.***:554/stream1

<?xml version="1.0" encoding="UTF-8"?>

from ffmpegcv.stream_info import get_info ***@***.***:554/stream1') [hangs, no reply] However, with cv2 (note, I have OPENCV_VIDEOIO_DEBUG=TRUE) Best, B >>> import cv2 >>> cap = ***@***.***:554/stream1') [ ***@***.*** global cap.cpp:130 open VIDEOIO(FFMPEG): trying capture ***@***.***:554/stream1' ... [ ***@***.*** global cap.cpp:142 open VIDEOIO(FFMPEG): created, isOpened=1 From: ChenXF ***@***.***> Sent: Thursday, August 10, 2023 13:50 To: chenxinfeng4/ffmpegcv ***@***.***> Cc: bertelschmitt ***@***.***>; Author ***@***.***> Subject: Re: [chenxinfeng4/ffmpegcv] .isOpened() (Issue #18) Can you run the code below and post the result? ffprobe -v quiet -print_format xml -select_streams v:0 -show_format -show_streams rtsp://YOURSTREAM And this. from ffmpegcv.stream_info import get_info get_info('rtsp://YOURSTREAM') — Reply to this email directly, view it on GitHub , or unsubscribe . You are receiving this because you authored the thread. Message ID: ***@***.*** ***@***.***> >
bertelschmitt commented 1 year ago

Also, just in case you need it:

===================================================================================

ffprobe -v debug -print_format xml -select_streams v:0 -show_format -show_streams @.***:554/stream1

ffprobe version 6.0 Copyright (c) 2007-2023 the FFmpeg developers

built with gcc 9 (Ubuntu 9.5.0-1ubuntu1~22.04)

configuration: --enable-shared --enable-nonfree --enable-gpl --enable-libx264 --enable-libx265 --enable-libvpx --enable-zlib --enable-gpl --disable-stripping --enable-gnutls --enable-libass --enable-libbs2b --enable-libcodec2 --enable-libdav1d --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-libshine --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-omx --enable-opencl --enable-libflite --enable-opengl --enable-libbluray --enable-sdl2 --enable-librsvg --enable-libmfx --enable-libdrm --enable-libx264 --enable-shared

libavutil 58. 2.100 / 58. 2.100

libavcodec 60. 3.100 / 60. 3.100

libavformat 60. 3.100 / 60. 3.100

libavdevice 60. 1.100 / 60. 1.100

libavfilter 9. 3.100 / 9. 3.100

libswscale 7. 1.100 / 7. 1.100

libswresample 4. 10.100 / 4. 10.100

libpostproc 57. 1.100 / 57. 1.100

<?xml version="1.0" encoding="UTF-8"?>

[tcp @ 0x55bb381b1b40] No default whitelist set [tcp @ 0x55bb381b1b40] Original list of addresses: [tcp @ 0x55bb381b1b40] Address 192.168.1.33 port 554 [tcp @ 0x55bb381b1b40] Interleaved list of addresses: [tcp @ 0x55bb381b1b40] Address 192.168.1.33 port 554 [tcp @ 0x55bb381b1b40] Starting connection attempt to 192.168.1.33 port 554 [tcp @ 0x55bb381b1b40] Successfully connected to 192.168.1.33 port 554 [rtsp @ 0x55bb381aeec0] SDP: v=0 o=- 1690362517264725 1 IN IP4 192.168.1.33 s=stream server i=stream1 t=0 0 a=tool:LIVE555 Streaming Media v2014.03.25 a=type:broadcast a=control:* a=range:npt=0- a=x-qt-text-nam:stream server a=x-qt-text-inf:stream1 m=video 0 RTP/AVP 96 c=IN IP4 0.0.0.0 b=AS:409 a=rtpmap:96 H264/90000 a=fmtp:96 packetization-mode=1;profile-level-id=4D401F;sprop-parameter-sets=Z01AH5WgFAFumwE=,aO48gA== a=Media_header:MEDIAINFO=494D4B48010100000400010010710110401F000000FA000000000000000000000000000000000000 a=framerate:8 a=framesize:96 1280-720 a=control:trackID=1 Failed to parse interval end specification '' [rtsp @ 0x55bb381aeec0] video codec set to: h264 [rtsp @ 0x55bb381aeec0] RTP Packetization Mode: 1 [rtsp @ 0x55bb381aeec0] RTP Profile IDC: 4d Profile IOP: 40 Level: 1f [rtsp @ 0x55bb381aeec0] Extradata set to 0x55bb381b1ae0 (size: 23) [rtp @ 0x55bb381b4100] No default whitelist set [udp @ 0x55bb381b4d40] No default whitelist set [udp @ 0x55bb381b4d40] end receive buffer size reported is 425984 [udp @ 0x55bb381b43c0] No default whitelist set [udp @ 0x55bb381b43c0] end receive buffer size reported is 425984 [rtsp @ 0x55bb381aeec0] setting jitter buffer size to 500 [rtsp @ 0x55bb381aeec0] hello state=0 Failed to parse interval end specification '' [h264 @ 0x55bb381b3a40] nal_unit_type: 7(SPS), nal_ref_idc: 3 [h264 @ 0x55bb381b3a40] nal_unit_type: 8(PPS), nal_ref_idc: 3 [h264 @ 0x55bb381b3a40] Decoding VUI ============================================================================ Best, B From: ***@***.*** ***@***.***> Sent: Thursday, August 10, 2023 15:52 To: 'chenxinfeng4/ffmpegcv' ***@***.***> Subject: RE: [chenxinfeng4/ffmpegcv] .isOpened() (Issue #18) ffprobe -v quiet -print_format xml -select_streams v:0 -show_format -show_streams ***@***.***:554/stream1 from ffmpegcv.stream_info import get_info ***@***.***:554/stream1') [hangs, no reply] However, with cv2 (note, I have OPENCV_VIDEOIO_DEBUG=TRUE) Best, B >>> import cv2 >>> cap = ***@***.***:554/stream1') [ ***@***.*** global cap.cpp:130 open VIDEOIO(FFMPEG): trying capture ***@***.***:554/stream1' ... [ ***@***.*** global cap.cpp:142 open VIDEOIO(FFMPEG): created, isOpened=1 From: ChenXF ***@***.*** ***@***.***> > Sent: Thursday, August 10, 2023 13:50 To: chenxinfeng4/ffmpegcv ***@***.*** ***@***.***> > Cc: bertelschmitt ***@***.*** ***@***.***> >; Author ***@***.*** ***@***.***> > Subject: Re: [chenxinfeng4/ffmpegcv] .isOpened() (Issue #18) Can you run the code below and post the result? ffprobe -v quiet -print_format xml -select_streams v:0 -show_format -show_streams rtsp://YOURSTREAM And this. from ffmpegcv.stream_info import get_info get_info('rtsp://YOURSTREAM') — Reply to this email directly, view it on GitHub , or unsubscribe . You are receiving this because you authored the thread. Message ID: ***@***.*** ***@***.***> >
bertelschmitt commented 1 year ago

By the way: The rtsp streams produced by IP cameras (I have a few) are notoriously buggy, they often report bogus capabilities, and require a lot of error checking. I learned not to take anything for granted.

B

chenxinfeng4 commented 1 year ago

The response from the ffprobe is abnormal. I do test in the rtsp. It should be like

$ ffprobe -v debug  -print_format xml -select_streams v:0 -show_format -show_streams rtsp://chenxinfeng.com:8554/stream400x300_mpeg4

<?xml version="1.0" encoding="UTF-8"?>
<ffprobe>
...
Input #0, rtsp, from 'rtsp://chenxinfeng.com:8554/stream400x300_mpeg4':
  Metadata:
    title           : Stream
  Duration: N/A, start: 0.252456, bitrate: N/A
    Stream #0:0, 2, 1/90000: Video: mpeg4 (Simple Profile), 1 reference frame, yuv420p(left), 400x300 [SAR 1:1 DAR 4:3], 0/1, 5 tbr, 90k tbn, 5 tbc
    <streams>
        <stream index="0" codec_name="mpeg4" codec_long_name="MPEG-4 part 2" profile="Simple Profile" codec_type="video" codec_time_base="0/1" codec_tag_string="[0][0][0][0]" codec_tag="0x0000" width="400" height="300" coded_width="400" coded_height="300" closed_captions="0" has_b_frames="0" sample_aspect_ratio="1:1" display_aspect_ratio="4:3" pix_fmt="yuv420p" level="1" chroma_location="left" refs="1" quarter_sample="false" divx_packed="false" r_frame_rate="5/1" avg_frame_rate="0/0" time_base="1/90000" start_pts="22721" start_time="0.252456">
            <disposition default="0" dub="0" original="0" comment="0" lyrics="0" karaoke="0" forced="0" hearing_impaired="0" visual_impaired="0" clean_effects="0" attached_pic="0" timed_thumbnails="0"/>
        </stream>
    </streams>

    <format filename="rtsp://chenxinfeng.com:8554/stream400x300_mpeg4" nb_streams="1" nb_programs="0" format_name="rtsp" format_long_name="RTSP input" start_time="0.252456" probe_score="100">
        <tag key="title" value="Stream"/>
    </format>
</ffprobe>