FFmpegFrameGrabber works only for few seconds.

[DrDobry]

I'm having some trouble with FFmpegFrameGrabber.grabImage() method, it only works for about 180 frames or so. I'm using Tenvis TH-671 cameras rtsp link which works just fine in VLC player, also when I use same code for other camera with rtsp link it works just fine. To clarify program gets stuck in grabImage() and cant get new frame (also tried grab instead grabImage), not sure what the problem is, here's sample code:

frameGrabber = new FFmpegFrameGrabber("rtsp://ip/11");
frameGrabber.start();
Frame frame;
BufferedImage img;
for(int i=0;i<1000;i++){
    Main.frameGrabber.flush();
    frame = Main.frameGrabber.grab();
    img = new Java2DFrameConverter().convert(frame);
}

Only message in console that I get, that I don't get with other camera, is:

[swscaler @ 0x60e00030] No accelerated colorspace conversion found from yuv420p to bgr24.

[saudet]

Try to set the log to "trace" level...

[DrDobry]

Thanks for the reply, Can you please elaborate or write code that does that.

[DrDobry]

After some debuging I've got this (it gets stuck here after 150ish iterations its different every time but its always +-10) :

this is with grabImage() method, and the line of code that from picture (783): if (av_read_frame(oc, pkt) < 0) { worth to mention that it can get stuck at the same line of code with flush().

Looks like some kind of deadlock to me, here is picture of other threads:

Is it maybe some kind of timeout since it gets stuck consistently at about same time, how do i set it, and why is it different for the other camera? If not how to get out of that deadlock, can you offer any help?

[saudet]

FFmpeg context objects are not thread-safe. You shouldn't be using the same FFmpegFrameGrabber from multiple threads, if that's what you're trying to do here.

[DrDobry]

All of my code for FrameGrabber is in single thread (main atm (Thread [main]), not sure what makes other threads, and yes i remember getting fatal errors when i tried to make separate threads for flush and grab a while back. Anyway not the case now.

[saudet]

What are you referring to by "flush"'?

[DrDobry]

FFmpegFrameGrabber frameGrabber; frameGrabber.flush(); I added flush with the other camera to sync the stream, without flush delay is adding up.

[saudet]

That indicates that you're not grabbing the images fast enough. You should reduce the FPS of your camera.

Anyway, call av_log_set_level(AV_LOG_TRACE) and check the last messages that you get.

[DrDobry]

Thank you for taking the time, I got lowest settings I can set, 5 FPS and 128 bit rate (much much lower than the other cam), sorry I'm not that seasoned programmer, how exactly do I call av_log_set_level(AV_LOG_TRACE)

[DrDobry]

Ok, sorry for dumb question, for anyone else who might be having same problem,I needed to put static import import static org.bytedeco.javacpp.avutil.* to use av_log_set_level(AV_LOG_TRACE)

Here's the relevant part of the code just in case:

        av_log_set_level(AV_LOG_TRACE);
        frameGrabber=new FFmpegFrameGrabber("rtsp://192.168.2.239/11");
        frameGrabber.start();

        Frame frame;
        for(int i=0;i<1000;i++){
            if (i % 20 == 0)
                System.out.println(i);
            frameGrabber.flush();
            frame = frameGrabber.grabImage();
            BufferedImage img = new Java2DFrameConverter().convert(frame);
            ImageIO.write(img, "jpeg", new File("FrameFolderTmp/snapshot_" + i + ".jpg"));
//          System.out.print('.');
        }

as for the log, i added printing iterator every for every 20 iterations, log is a bit chaotic (paste after 140th iteration):

[h264 @140
 000000001b4fc060] nal_unit_type: 1, nal_ref_idc: 3
[h264 @ 000000001b4fc060] nal_unit_type: 1, nal_ref_idc: 1
[h264 @ 000000001b4fc060] nal_unit_type: 1, nal_ref_idc: 3
[h264 @ 000000001b4fc060] nal_unit_type: 1, nal_ref_idc: 1
[h264 @ 000000001b4fc060] nal_unit_type: 1, nal_ref_idc: 3
[h264 @ 000000001b4fc060] nal_unit_type: 1, nal_ref_idc: 1
[h264 @ 000000001b4fc060] nal_unit_type: 1, nal_ref_idc: 3
[h264 @ 000000001b4fc060] nal_unit_type: 5, nal_ref_idc: 3
[h264 @ 000000001b4fc060] nal_unit_type: 1, nal_ref_idc: 1
[h264 @ 000000001b4fc060] nal_unit_type: 1, nal_ref_idc: 3
[h264 @ 000000001b4fc060] nal_unit_type: 1, nal_ref_idc: 1
[h264 @ 000000001b4fc060] nal_unit_type: 1, nal_ref_idc: 3
[h264 @ 000000001b4fc060] nal_unit_type: 1, nal_ref_idc: 1
[h264 @ 000000001b4fc060] nal_unit_type: 1, nal_ref_idc: 3
[h264 @ 000000001b4fc060] nal_unit_type: 1, nal_ref_idc: 1
[rtsp @ 0000000019f45f00] Sending:
GET_PARAMETER rtsp://192.168.2.239:554/11/ RTSP/1.0

CSeq: 6

User-Agent: Lavf57.56.100

Session: 111085237568107

--
[rtsp @ 0000000019f45f00] ret=1 c=52 [R]
[rtsp @ 0000000019f45f00] ret=1 c=54 [T]
[rtsp @ 0000000019f45f00] ret=1 c=53 [S]
[rtsp @ 0000000019f45f00] ret=1 c=50 [P]
[rtsp @ 0000000019f45f00] ret=1 c=2f [/]
[rtsp @ 0000000019f45f00] ret=1 c=31 [1]
[rtsp @ 0000000019f45f00] ret=1 c=2e [.]
[rtsp @ 0000000019f45f00] ret=1 c=30 [0]
[rtsp @ 0000000019f45f00] ret=1 c=20 [ ]
[rtsp @ 0000000019f45f00] ret=1 c=32 [2]
[rtsp @ 0000000019f45f00] ret=1 c=30 [0]
[rtsp @ 0000000019f45f00] ret=1 c=30 [0]
[rtsp @ 0000000019f45f00] ret=1 c=20 [ ]
[rtsp @ 0000000019f45f00] ret=1 c=4f [O]
[rtsp @ 0000000019f45f00] ret=1 c=4b [K]
[rtsp @ 0000000019f45f00] ret=1 c=0d [
]
[rtsp @ 0000000019f45f00] ret=1 c=0a [
]
[rtsp @ 0000000019f45f00] line='RTSP/1.0 200 OK'
[rtsp @ 0000000019f45f00] ret=1 c=53 [S]
[rtsp @ 0000000019f45f00] ret=1 c=65 [e]
[rtsp @ 0000000019f45f00] ret=1 c=72 [r]
[rtsp @ 0000000019f45f00] ret=1 c=76 [v]
[rtsp @ 0000000019f45f00] ret=1 c=65 [e]
[rtsp @ 0000000019f45f00] ret=1 c=72 [r]
[rtsp @ 0000000019f45f00] ret=1 c=3a [:]
[rtsp @ 0000000019f45f00] ret=1 c=20 [ ]
[rtsp @ 0000000019f45f00] ret=1 c=48 [H]
[rtsp @ 0000000019f45f00] ret=1 c=69 [i]
[rtsp @ 0000000019f45f00] ret=1 c=49 [I]
[rtsp @ 0000000019f45f00] ret=1 c=70 [p]
[rtsp @ 0000000019f45f00] ret=1 c=63 [c]
[rtsp @ 0000000019f45f00] ret=1 c=61 [a]
[rtsp @ 0000000019f45f00] ret=1 c=6d [m]
[rtsp @ 0000000019f45f00] ret=1 c=2f [/]
[rtsp @ 0000000019f45f00] ret=1 c=56 [V]
[rtsp @ 0000000019f45f00] ret=1 c=31 [1]
[rtsp @ 0000000019f45f00] ret=1 c=30 [0]
[rtsp @ 0000000019f45f00] ret=1 c=30 [0]
[rtsp @ 0000000019f45f00] ret=1 c=52 [R]
[rtsp @ 0000000019f45f00] ret=1 c=30 [0]
[rtsp @ 0000000019f45f00] ret=1 c=30 [0]
[rtsp @ 0000000019f45f00] ret=1 c=33 [3]
[rtsp @ 0000000019f45f00] ret=1 c=20 [ ]
[rtsp @ 0000000019f45f00] ret=1 c=56 [V]
[rtsp @ 0000000019f45f00] ret=1 c=6f [o]
[rtsp @ 0000000019f45f00] ret=1 c=64 [d]
[rtsp @ 0000000019f45f00] ret=1 c=53 [S]
[rtsp @ 0000000019f45f00] ret=1 c=65 [e]
[rtsp @ 0000000019f45f00] ret=1 c=72 [r]
[rtsp @ 0000000019f45f00] ret=1 c=76 [v]
[rtsp @ 0000000019f45f00] ret=1 c=65 [e]
[rtsp @ 0000000019f45f00] ret=1 c=72 [r]
[rtsp @ 0000000019f45f00] ret=1 c=2f [/]
[rtsp @ 0000000019f45f00] ret=1 c=31 [1]
[rtsp @ 0000000019f45f00] ret=1 c=2e [.]
[rtsp @ 0000000019f45f00] ret=1 c=30 [0]
[rtsp @ 0000000019f45f00] ret=1 c=2e [.]
[rtsp @ 0000000019f45f00] ret=1 c=30 [0]
[rtsp @ 0000000019f45f00] ret=1 c=0d [
]
[rtsp @ 0000000019f45f00] ret=1 c=0a [
]
[rtsp @ 0000000019f45f00] line='Server: HiIpcam/V100R003 VodServer/1.0.0'
[rtsp @ 0000000019f45f00] ret=1 c=43 [C]
[rtsp @ 0000000019f45f00] ret=1 c=73 [s]
[rtsp @ 0000000019f45f00] ret=1 c=65 [e]
[rtsp @ 0000000019f45f00] ret=1 c=71 [q]
[rtsp @ 0000000019f45f00] ret=1 c=3a [:]
[rtsp @ 0000000019f45f00] ret=1 c=20 [ ]
[rtsp @ 0000000019f45f00] ret=1 c=36 [6]
[rtsp @ 0000000019f45f00] ret=1 c=0d [
]
[rtsp @ 0000000019f45f00] ret=1 c=0a [
]
[rtsp @ 0000000019f45f00] line='Cseq: 6'
[rtsp @ 0000000019f45f00] ret=1 c=53 [S]
[rtsp @ 0000000019f45f00] ret=1 c=65 [e]
[rtsp @ 0000000019f45f00] ret

Edit: I was looking at logs a bit and found the line: [rtsp @ 0000000019f05f00] line='Session: 162916216685851; timeout=60' Decided to check time at which the program stops, its not 60s but it is always 30s at the end of last iteration, more specifically its less than 30 at second to last and over 30 at last one (I started the timer after making instance of FFmpegFrameGrabber and print time after ImageIO.write). I will try to test further if its 100% timed event, and try to find a way to set that timer, if you see this in meantime please tell me how to set that timeout.

Edit: I'm not sure if its timed event, when I add for loop for saving image time is no longer 30s (in some cases) but I'm confused, for sake of clarity I put:

for(int j=0;j<x;j++)
                ImageIO.write(img, "jpeg", new File("FrameFolderTmp/snapshot_" + i + ".jpg"));

if x = 1,10,25,50 it takes exactly 30s (less iterations in first for loop ofc,25 if 100, 37 if 50, 145 if 1) and for x=75 it takes ~32s, x=100 ~39s. Also worth to mention that flush and grabImage take significantly less time for bigger x. So all in all I can't really say it does or doesn't have to do with some timer...

[saudet]

If you're not able to save all images, you should drop them somehow, or the upstream server/camera might decide to stop sending you new images. This is probably what is happening here.

[DrDobry]

no, that's not the problem, I only put ImageIO.write to see if it is time related issue, works the same way without that line of code. The thing that confuses me the most is the fact that it works just fine in vlc and when i use the command in java it works for 30s (most of the time). Only solution I can come up with is to restart the stream when it reaches 30s mark, but that's terrible solution.

[saudet]

So it works fine with VLC, but what about ffplay?

[DrDobry]

I don't know about ffplay, can I test it from eclipse with javaCV or do I need to play around with ffmpeg build in cmd? Edit: I figured it out with cmd, camera is not available at the moment I will try it with ffplay when I can, and let you know if it works.

[DrDobry]

It doesn't work with ffmpegs ffplay, or i should say it works for about 30s and then freezes, aq and vq become =0... So do you have any idea if this can be fixed, I know this makes it problem with ffmpeg but any advice would be appreciated.

[sezertt]

Hi saudet you got me wrong with parameters, I also tried the same parameters which I told you about

If I add tcp to white list, again it wont play... By the way thank you for your answers

But it doesn't matter as you said it wont work with ffplay either.

[saudet]

@sezertt Right, so please file a ticket upstream about this so they can fix it in FFmpeg! Thanks

[saudet]

Any updates? Please try again with JavaCV 1.4 and FFmpeg 3.4.1. It might be fixed by now.

[DrDobry]

Nah, I gave up on that some time ago, I don't even have that camera anymore.