search

angelcam / rust-ac-ffmpeg

Simple and safe Rust interface for FFmpeg libraries.
MIT License
197 stars 33 forks source link

Demux srt mpegts stream through ffmpeg #34

Closed niklaskorz closed 3 years ago

niklaskorz commented 3 years ago

(duplicate of https://github.com/russelltg/srt-rs/issues/124 in case this is more related to ffmpeg than SRT)

I'm currently trying to demux an mpegts stream received through https://github.com/russelltg/srt-rs to mux it as a DASH and HLS livestream, all using ac_ffmpeg. However, ac_ffmpeg expects the Read trait to be implemented for input, which I have tried to somewhat mimick:

struct ByteReceiver {
    rx: Receiver<bytes::Bytes>,
    prev: Option<(bytes::Bytes, usize)>,
}

impl ByteReceiver {
    fn new(rx: Receiver<bytes::Bytes>) -> Self {
        Self { rx, prev: None }
    }
}

impl Read for ByteReceiver {
    fn read(&mut self, buf: &mut [u8]) -> std::io::Result<usize> {
        if let Some(ref mut prev) = self.prev {
            let limit = std::cmp::min(prev.0.len() - prev.1, buf.len());
            buf[..limit].copy_from_slice(&prev.0[prev.1..prev.1 + limit]);
            if prev.1 + limit < prev.0.len() {
                prev.1 += limit;
            } else {
                self.prev = None
            }
            Ok(limit)
        } else if let Ok(bytes) = self.rx.recv() {
            let limit = std::cmp::min(bytes.len(), buf.len());
            buf[..limit].copy_from_slice(&bytes[..limit]);
            if buf.len() < bytes.len() {
                self.prev = Some((bytes, buf.len()));
            }
            Ok(limit)
        } else {
            Ok(0)
        }
    }
}

async fn handle_socket(mut socket: SrtSocket, tx: Sender<bytes::Bytes>) -> Result<()> {
    let socket_id = socket.settings().remote_sockid.0;
    let client_desc = format!(
        "(ip_port: {}, sockid: {}, stream_id: {:?})",
        socket.settings().remote,
        socket_id,
        socket.settings().stream_id,
    );
    println!("New client connected: {}", client_desc);

    let mut count: i32 = 0;
    while let Some(packet) = socket.next().await {
        let (_instant, bytes) = packet?;
        tx.send(bytes)?;
        count += 1;
    }

    println!(
        "Client {} disconnected, received {:?} packets",
        client_desc, count
    );
    Ok(())
}

The full code can be found here: https://gist.github.com/niklaskorz/0e623c667b76dc30fa872282331033e4

Unfortunately, this results in lots of error messages on ffmpeg's side:

$ cargo run
   Compiling rust-live v0.1.0 (/Users/niklaskorz/Development/alugha/old-rust-live)
    Finished dev [unoptimized + debuginfo] target(s) in 2.24s
     Running `target/debug/rust-live`
New client connected: (ip_port: 192.168.0.91:37878, sockid: 1002830551, stream_id: Some("thestream"))
[aac @ 0x7fb1da82be00] Gain control is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[aac @ 0x7fb1da82be00] Input buffer exhausted before END element found
[h264 @ 0x7fb1da82b200] concealing 579 DC, 579 AC, 579 MV errors in P frame
[h264 @ 0x7fb1da82b200] error while decoding MB 110 63, bytestream -5
[h264 @ 0x7fb1da82b200] concealing 539 DC, 539 AC, 539 MV errors in P frame
[mpegts @ 0x7fb1dc809200] PES packet size mismatch
[mpegts @ 0x7fb1dc809200] Packet corrupt (stream = 0, dts = 23772).
[h264 @ 0x7fb1da82b200] concealing 587 DC, 587 AC, 587 MV errors in P frame
[h264 @ 0x7fb1da82b200] Invalid NAL unit 0, skipping.
[h264 @ 0x7fb1da82b200] error while decoding MB 109 5, bytestream -11
[h264 @ 0x7fb1da82b200] concealing 7500 DC, 7500 AC, 7500 MV errors in P frame
[mpegts @ 0x7fb1dc809200] PES packet size mismatch
[mpegts @ 0x7fb1dc809200] Packet corrupt (stream = 0, dts = 35755).
[mpegts @ 0x7fb1dc809200] PES packet size mismatch
[mpegts @ 0x7fb1dc809200] Packet corrupt (stream = 0, dts = 41749).
[mpegts @ 0x7fb1dc809200] PES packet size mismatch
[mpegts @ 0x7fb1dc809200] Packet corrupt (stream = 0, dts = 47747).
[mpegts @ 0x7fb1dc809200] PES packet size mismatch
[mpegts @ 0x7fb1dc809200] Packet corrupt (stream = 0, dts = 53709).
[mpegts @ 0x7fb1dc809200] PES packet size mismatch
[mpegts @ 0x7fb1dc809200] Packet corrupt (stream = 0, dts = 59707).
[mpegts @ 0x7fb1dc809200] PES packet size mismatch
[mpegts @ 0x7fb1dc809200] Packet corrupt (stream = 1, dts = 65072).
[mpegts @ 0x7fb1dc809200] Packet corrupt (stream = 1, dts = 69252).
[mpegts @ 0x7fb1dc809200] Packet corrupt (stream = 1, dts = 75521).
Stream #0:
  duration: None
  time base: 1 / 90000
  type: video
  codec: h264
  width: 1920
  height: 1080
  pixel format: yuv420p
Stream #1:
  duration: None
  time base: 1 / 90000
  type: audio
  codec: aac
  sample format: fltp
  sample rate: 44100
  channels: 1
  bitrate: 217383
[dash @ 0x7fb1dc009600] No bit rate set for stream 0
[dash @ 0x7fb1dc009600] Opening 'segments/init-stream0.m4s' for writing
[dash @ 0x7fb1dc009600] Opening 'segments/init-stream1.m4s' for writing
[dash @ 0x7fb1dc009600] Opening 'segments/chunk-stream1-00001.m4s.tmp' for writing
[dash @ 0x7fb1dc009600] Opening 'segments/chunk-stream0-00001.m4s.tmp' for writing
[aac_adtstoasc @ 0x7fb1db226100] Input packet too small
[dash @ 0x7fb1dc009600] Error applying bitstream filters to an output packet for stream #1: Invalid data found when processing input
[dash @ 0x7fb1dc009600] Opening 'segments/dash.mpd.tmp' for writing
[dash @ 0x7fb1dc009600] Opening 'segments/media_0.m3u8.tmp' for writing
[dash @ 0x7fb1dc009600] Opening 'segments/media_1.m3u8.tmp' for writing
[dash @ 0x7fb1dc009600] Opening 'segments/master.m3u8.tmp' for writing
Error in socket handler: sending on a closed channel
New client connected: (ip_port: 192.168.0.91:45051, sockid: 1002830550, stream_id: Some("thestream"))
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 0).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 2998).
[h264 @ 0x7fb1da822200] top block unavailable for requested intra mode -1
[h264 @ 0x7fb1da822200] error while decoding MB 58 0, bytestream 34485
[h264 @ 0x7fb1da822200] concealing 8151 DC, 8151 AC, 8151 MV errors in I frame
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 5998).
[h264 @ 0x7fb1da822200] top block unavailable for requested intra mode
[h264 @ 0x7fb1da822200] error while decoding MB 53 0, bytestream 13685
[h264 @ 0x7fb1da822200] concealing 8156 DC, 8156 AC, 8156 MV errors in P frame
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 11995).
[h264 @ 0x7fb1da822200] concealing 7782 DC, 7782 AC, 7782 MV errors in P frame
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 17993).
[h264 @ 0x7fb1da822200] concealing 7478 DC, 7478 AC, 7478 MV errors in P frame
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 20994).
[h264 @ 0x7fb1da822200] top block unavailable for requested intra mode -1
[h264 @ 0x7fb1da822200] error while decoding MB 100 0, bytestream 10255
[h264 @ 0x7fb1da822200] concealing 8109 DC, 8109 AC, 8109 MV errors in P frame
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 23992).
[h264 @ 0x7fb1da822200] top block unavailable for requested intra mode -1
[h264 @ 0x7fb1da822200] error while decoding MB 71 0, bytestream 7655
[h264 @ 0x7fb1da822200] concealing 8138 DC, 8138 AC, 8138 MV errors in P frame
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 35986).
[h264 @ 0x7fb1da822200] concealing 7900 DC, 7900 AC, 7900 MV errors in P frame
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 38986).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 44984).
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 47982).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 53980).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 56979).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 59979).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 62978).
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 65976).
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 71974).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 80971).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 83971).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 86971).
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 89967).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 95966).
Stream #0:
  duration: None
  time base: 1 / 90000
  type: video
  codec: h264
  width: 1920
  height: 1080
  pixel format: yuv420p
Stream #1:
  duration: None
  time base: 1 / 90000
  type: audio
  codec: aac
  sample format: fltp
  sample rate: 44100
  channels: 1
  bitrate: 51367
[dash @ 0x7fb1da8a0000] No bit rate set for stream 0
[dash @ 0x7fb1da8a0000] Opening 'segments/init-stream0.m4s' for writing
[dash @ 0x7fb1da8a0000] Opening 'segments/init-stream1.m4s' for writing
[dash @ 0x7fb1da8a0000] Opening 'segments/chunk-stream1-00001.m4s.tmp' for writing
[dash @ 0x7fb1da8a0000] Opening 'segments/chunk-stream0-00001.m4s.tmp' for writing
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 98971).
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = NOPTS).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 104963).
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 107961).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 119965).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 122959).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 125956).
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 128954).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 134952).
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 137952).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 143954).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 146949).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 149955).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 152946).
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = NOPTS).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 161943).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 164944).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 167942).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 170941).
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 176939).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 185947).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 188934).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 191933).
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 194932).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 200930).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 203929).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 206932).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 209926).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 212925).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 215924).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 218933).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 221923).
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 224921).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 230919).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 233919).
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 236918).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 248913).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 251913).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 254913).
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 257910).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 266907).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 269906).
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 272904).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 277529).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 280528).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 283527).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 286526).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 289525).
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 292524).
[mpegts @ 0x7fb1da81ee00] PES packet size mismatch
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 1, dts = 290368).
[mpegts @ 0x7fb1da81ee00] Packet corrupt (stream = 0, dts = 298522).
[aac_adtstoasc @ 0x7fb1da4606c0] Error parsing ADTS frame header!
[dash @ 0x7fb1da8a0000] Error applying bitstream filters to an output packet for stream #1: Invalid data found when processing input
[dash @ 0x7fb1da8a0000] Opening 'segments/dash.mpd.tmp' for writing
[dash @ 0x7fb1da8a0000] Opening 'segments/media_0.m3u8.tmp' for writing
[dash @ 0x7fb1da8a0000] Opening 'segments/media_1.m3u8.tmp' for writing
[dash @ 0x7fb1da8a0000] Opening 'segments/master.m3u8.tmp' for writing
Error in socket handler: sending on a closed channel

Is there anything wrong with my approach of using channels and buffering?

niklaskorz commented 3 years ago

The problem was on srt-rs's side, so I will close this issue