how to write the audio into fmp4?

[lrc0]

Hi,I am a newcomer doing audio and video processing.Then I chose this library,but now I have some problems:

The demo only shows how to write video to fmp4, I can't find any demo that writes audio and video to fmp4 together.

i try to use the method:

func (this *FMP4Muxer) AddPacket(packet *AVPacket.MediaPacket) (err error)

The video is correct,but there is always something wrong with the audio.

this is my video code(same as the demo):

func (muxer *Muxer) writeVideoFrame(data []byte, timestamp uint32, naluType int) (err error) {
    tag := &AVPacket.MediaPacket{}
    tag.PacketType = AVPacket.AV_PACKET_TYPE_VIDEO
    tag.TimeStamp = int64(timestamp)
    tag.Data = make([]byte, len(data)+5+4)
    if naluType == h264.NALU_IDR {
        tag.Data[0] = 0x17
    } else {
        tag.Data[0] = 0x27
    }

    tag.Data[1] = 1
    cts := 0
    tag.Data[2] = byte((cts >> 16) & 0xff)
    tag.Data[3] = byte((cts >> 8) & 0xff)
    tag.Data[4] = byte((cts >> 0) & 0xff)
    nalSize := len(data)
    tag.Data[5] = byte((nalSize >> 24) & 0xff)
    tag.Data[6] = byte((nalSize >> 16) & 0xff)
    tag.Data[7] = byte((nalSize >> 8) & 0xff)
    tag.Data[8] = byte((nalSize >> 0) & 0xff)
    copy(tag.Data[9:], data)
    //add into fmp4
    err = muxer.fmp4.AddPacket(tag)
    if err != nil {
        return
    }
    _, mdat, _, _, err := muxer.fmp4.Flush()
    if err != nil {
        return
    }

    pkt := av.NewPacket()
    pkt.Type = av.PACKET_FMP4
    pkt.VideoEncode = av.VIDEO_H264
    pkt.Stream = mdat
    pkt.Ext = muxer.sps
    pkt.IsKey = (naluType == h264.NALU_IDR)
    return muxer.Filter.Write(pkt)
}

this is my audio code(follow the code to process AAC frames):

func (muxer *Muxer) parseAACFrame(data []byte, timestamp int64) []byte {

    tag := &AVPacket.MediaPacket{}
    if !muxer.headerDecode {
        asconf := AAC.AACGetConfig(data)
        if asconf.ObjectType() == 0 || asconf.SampleRate() == 0 ||
            asconf.Channel() == 0 {
            return nil
        }
        fmt.Printf("======> asconf: %+v\n", asconf)

        timestamp = timestamp * int64(8000) / int64(1000)
        muxer.headerDecode = true
        tag.PacketType = AVPacket.AV_PACKET_TYPE_AUDIO
        tag.TimeStamp = timestamp
        tag.Data = make([]byte, 2+len(data))
        tag.Data[0] = 0xaf
        tag.Data[1] = 0
        copy(tag.Data[2:], data)
        muxer.fmp4.SetAudioHeader(tag)
        return nil
    }

    timestamp = timestamp * int64(8000) / int64(1000)
    tag.PacketType = AVPacket.AV_PACKET_TYPE_AUDIO
    tag.Data = make([]byte, 2+len(data))
    tag.TimeStamp = timestamp
    tag.Data[0] = 0xaf
    tag.Data[1] = 1
    copy(tag.Data[2:], data)

         //add into fmp4
    err := muxer.fmp4.AddPacket(tag)
    if err != nil {
        logger.Error(err.Error())
        return nil
    }

    _, mdat, _, _, err := muxer.fmp4.Flush()
    if err != nil {
        logger.Error(err.Error())
        return nil
    }
    return mdat
}

the result is video is correct, the audio is wrong. And i have no idea how to make it right. so please help me !!!

[widefire]

you can take a look https://github.com/widefire/websocketStreamServer/tree/master/DASH this project DASH use this library and work well

[shengsheng-yixi]

我想使用这个库把H264和AAC一起写入fmp4，请问你解决了吗，请教

你好，我是一个做音视频处理的新手，后来选择了这个库，但是现在遇到了一些问题：

演示只展示了如何将视频写入fmp4，我找不到任何将音频和视频一起写入fmp4的演示。

我尝试使用以下方法：

func (this *FMP4Muxer) AddPacket(packet *AVPacket.MediaPacket) (err error)

视频是正确的，但音频总是有问题。

这是我的视频代码（与演示相同）：

func (muxer *Muxer) writeVideoFrame(data []byte, timestamp uint32, naluType int) (err error) {
  tag := &AVPacket.MediaPacket{}
  tag.PacketType = AVPacket.AV_PACKET_TYPE_VIDEO
  tag.TimeStamp = int64(timestamp)
  tag.Data = make([]byte, len(data)+5+4)
  if naluType == h264.NALU_IDR {
      tag.Data[0] = 0x17
  } else {
      tag.Data[0] = 0x27
  }

  tag.Data[1] = 1
  cts := 0
  tag.Data[2] = byte((cts >> 16) & 0xff)
  tag.Data[3] = byte((cts >> 8) & 0xff)
  tag.Data[4] = byte((cts >> 0) & 0xff)
  nalSize := len(data)
  tag.Data[5] = byte((nalSize >> 24) & 0xff)
  tag.Data[6] = byte((nalSize >> 16) & 0xff)
  tag.Data[7] = byte((nalSize >> 8) & 0xff)
  tag.Data[8] = byte((nalSize >> 0) & 0xff)
  copy(tag.Data[9:], data)
  //add into fmp4
  err = muxer.fmp4.AddPacket(tag)
  if err != nil {
      return
  }
  _, mdat, _, _, err := muxer.fmp4.Flush()
  if err != nil {
      return
  }

  pkt := av.NewPacket()
  pkt.Type = av.PACKET_FMP4
  pkt.VideoEncode = av.VIDEO_H264
  pkt.Stream = mdat
  pkt.Ext = muxer.sps
  pkt.IsKey = (naluType == h264.NALU_IDR)
  return muxer.Filter.Write(pkt)
}

这是我的音频代码（按照代码处理 AAC 帧）：

func (muxer *Muxer) parseAACFrame(data []byte, timestamp int64) []byte {

  tag := &AVPacket.MediaPacket{}
  if !muxer.headerDecode {
      asconf := AAC.AACGetConfig(data)
      if asconf.ObjectType() == 0 || asconf.SampleRate() == 0 ||
          asconf.Channel() == 0 {
          return nil
      }
      fmt.Printf("======> asconf: %+v\n", asconf)

      timestamp = timestamp * int64(8000) / int64(1000)
      muxer.headerDecode = true
      tag.PacketType = AVPacket.AV_PACKET_TYPE_AUDIO
      tag.TimeStamp = timestamp
      tag.Data = make([]byte, 2+len(data))
      tag.Data[0] = 0xaf
      tag.Data[1] = 0
      copy(tag.Data[2:], data)
      muxer.fmp4.SetAudioHeader(tag)
      return nil
  }

  timestamp = timestamp * int64(8000) / int64(1000)
  tag.PacketType = AVPacket.AV_PACKET_TYPE_AUDIO
  tag.Data = make([]byte, 2+len(data))
  tag.TimeStamp = timestamp
  tag.Data[0] = 0xaf
  tag.Data[1] = 1
  copy(tag.Data[2:], data)

         //add into fmp4
  err := muxer.fmp4.AddPacket(tag)
  if err != nil {
      logger.Error(err.Error())
      return nil
  }

  _, mdat, _, _, err := muxer.fmp4.Flush()
  if err != nil {
      logger.Error(err.Error())
      return nil
  }
  return mdat
}

结果是视频正确，音频错误。我不知道如何使它正确。 所以请帮助我！！！

我想使用这个库把H264和AAC一起写入fmp4，请问你解决了吗，请教

[widefire]

https://github.com/panda-media/muxer-fmp4/blob/master/dashSlicer/dashSlicer.go,has example to mux aac

[shengsheng-yixi]

vedio.zip I'm a beginner of audio and video. I can't understand this example. Now I want to combine H264 and AAC files in the uploaded file to synthesize fmp4 video. How can I use it? Please, please

[widefire]

https://github.com/panda-media/muxer-fmp4/blob/master/example/dash/dashTest.go here is a easy example FlvFileToFMP4,this example read a flv file and save to fmp4,it's easy

[shengsheng-yixi]

Maybe you think it's simple, but I don't have any foundation. I'm not so powerful. I can't understand it. I don't know what process steps are required. I've been bothering this problem for a week. I want to read it from the code. Can you help me write it when you're free? Thank you very much