Closed libenli closed 1 year ago
The timestamps are probably wrong. You'll need to make sure they are correct.
The timestamps are probably wrong. You'll need to make sure they are correct.
Thanks,I try to add setTimestamp() refer to other issue, It seems to work,the video mixed will not be stuck after more than a minute; But as the playing time goes on, the time delay is getting bigger and bigger, from the first 6 seconds to more than 30 seconds, and the audio and video will get stuck every 10 seconds after five minutes. The playback will resume again after using VLC. Is the time stamp I set incorrect?
Here is the part of code which I changed:
while (true) {
frame0 = stream0.grabFrame();
frame1 = stream1.grabFrame();
if (frame0 != null) {
filter.push(0, frame0);
}
if (frame1 != null) {
filter.push(1, frame1);
}
if (frame0 == null || frame1 == null) {
if (err_index > 3) {
break;
}
err_index++;
continue;
}
if ((frame = filter.pull()) != null) {
long rt = recorder.getTimestamp();
if (frame.timestamp > rt) {
recorder.setTimestamp(frame.timestamp);
}
recorder.record(frame, filter.getPixelFormat());
}
}
You'll need to make sure the timestamps of all the frames stay close to each other, usually something like within a second.
Is it the original input frame or the filtered frame
You'll need to make sure the timestamps of all the frames stay close to each other, usually something like within a second.
Is it the original input frame or the filtered frame?Then compare it with the timestamp of the Record Frame?
Is it the original input frame or the filtered frame?Then compare it with the timestamp of the Record Frame?
I did said all the frames, input, output, filtered, all the frames, all of them.
I printed the timestamps of all frames and reset the video timestamps according to your instructions, but how do I set the audio? Now the result is that if the audio filter is shielded in the filter, the playback is normal, but if the audio filter is turned on at the same time, the video after merging will still be stuck after 3 minutes. Please take a look. Thanks a lot.
Here is the code:
while ((frame0 = stream0.grabFrame()) != null && (frame1 = stream1.grabFrame()) != null) {
filter.push(0, frame0);
filter.push(1, frame1);
long t0 = stream0.getTimestamp();
long t1 = stream1.getTimestamp();
while ((frame = filter.pull()) != null) {
long rt = recorder.getTimestamp();
long ft = frame.timestamp;
if (frame0.type == Frame.Type.VIDEO && frame1.type == Frame.Type.VIDEO) {
System.out.println("t0="+t0+" t1="+t1+" filter="+ft+" recorder="+rt + " type0="+frame0.type+ " type1="+frame1.type);
recorder.setTimestamp(t1);
recorder.record(frame, filter.getPixelFormat());
}
else if (frame0.type == Frame.Type.AUDIO && frame1.type == Frame.Type.AUDIO) {
recorder.record(frame, filter.getPixelFormat());
System.err.println("------t0="+t0+" t1="+t1+" filter="+ft+" recorder="+rt + " type0="+frame0.type+ " type1="+frame1.type);
}
}
}
Here is the log:
t0=61009000 t1=61010000 filter=60920000 recorder=60966667 type0=VIDEO type1=VIDEO
------t0=61127000 t1=61114000 filter=66385850 recorder=61033333 type0=AUDIO type1=AUDIO
------t0=61170000 t1=61157000 filter=66432290 recorder=61033333 type0=AUDIO type1=AUDIO
t0=61076000 t1=61076000 filter=61000000 recorder=61033333 type0=VIDEO type1=VIDEO
------t0=61191000 t1=61178000 filter=66455510 recorder=61100000 type0=AUDIO type1=AUDIO
------t0=61234000 t1=61221000 filter=66501950 recorder=61100000 type0=AUDIO type1=AUDIO
------t0=61298000 t1=61285000 filter=66571609 recorder=61100000 type0=AUDIO type1=AUDIO
------t0=61340000 t1=61327000 filter=66618049 recorder=61100000 type0=AUDIO type1=AUDIO
t0=61243000 t1=61243000 filter=61160000 recorder=61100000 type0=VIDEO type1=VIDEO
------t0=61362000 t1=61349000 filter=66641269 recorder=61266667 type0=AUDIO type1=AUDIO
------t0=61404000 t1=61391000 filter=66687709 recorder=61266667 type0=AUDIO type1=AUDIO
------t0=61426000 t1=61413000 filter=66710929 recorder=61266667 type0=AUDIO type1=AUDIO
------t0=61468000 t1=61455000 filter=66757369 recorder=61266667 type0=AUDIO type1=AUDIO
t0=61376000 t1=61376000 filter=61280000 recorder=61266667 type0=VIDEO type1=VIDEO
------t0=61490000 t1=61477000 filter=66780589 recorder=61400000 type0=AUDIO type1=AUDIO
------t0=61532000 t1=61519000 filter=66827029 recorder=61400000 type0=AUDIO type1=AUDIO
------t0=61575000 t1=61562000 filter=66873469 recorder=61400000 type0=AUDIO type1=AUDIO
t0=61476000 t1=61476000 filter=61400000 recorder=61400000 type0=VIDEO type1=VIDEO
------t0=61596000 t1=61583000 filter=66896689 recorder=61500000 type0=AUDIO type1=AUDIO
------t0=61639000 t1=61626000 filter=66943129 recorder=61500000 type0=AUDIO type1=AUDIO
t0=61543000 t1=61543000 filter=61440000 recorder=61500000 type0=VIDEO type1=VIDEO
------t0=61660000 t1=61647000 filter=66966349 recorder=61566667 type0=AUDIO type1=AUDIO
------t0=61703000 t1=61690000 filter=67012789 recorder=61566667 type0=AUDIO type1=AUDIO
t0=61609000 t1=61610000 filter=61520000 recorder=61566667 type0=VIDEO type1=VIDEO
------t0=61724000 t1=61711000 filter=67036009 recorder=61633333 type0=AUDIO type1=AUDIO
------t0=61767000 t1=61754000 filter=67082448 recorder=61633333 type0=AUDIO type1=AUDIO
------t0=61831000 t1=61818000 filter=67152108 recorder=61633333 type0=AUDIO type1=AUDIO
------t0=61874000 t1=61861000 filter=67198548 recorder=61633333 type0=AUDIO type1=AUDIO
t0=61776000 t1=61776000 filter=61680000 recorder=61633333 type0=VIDEO type1=VIDEO
------t0=61895000 t1=61882000 filter=67221768 recorder=61800000 type0=AUDIO type1=AUDIO
------t0=61938000 t1=61925000 filter=67268208 recorder=61800000 type0=AUDIO type1=AUDIO
t0=61843000 t1=61843000 filter=61760000 recorder=61800000 type0=VIDEO type1=VIDEO
------t0=61959000 t1=61946000 filter=67291428 recorder=61866667 type0=AUDIO type1=AUDIO
------t0=62002000 t1=61989000 filter=67337868 recorder=61866667 type0=AUDIO type1=AUDIO
------t0=62023000 t1=62010000 filter=67361088 recorder=61866667 type0=AUDIO type1=AUDIO
We can't set the timestamps for audio, you'll need to make sure to send the right amount of frames.
I revised the audio sampling rate of the filter and the recorder. It works normally. The timestamps of all frames of the print log are close. Thank you very much!
The two channels of rtmp, including audio and video, are combined and then pushed to rtmp, which is played using VLC. It is found that the video will be stuck after more than a minute, but there is mixed audio. What is the reason? Thank you
Here is the code: