AAC streamed byte packets and recordSamples

[danrossi]

I have a use case to get packets from a streaming server, from an RTMP encoder. And transcode to Ogg/Opus using recordSamples. It's similar to the microphone examples but different. Giving the recorder the raw AAC bytes produces a noisy output. Using the frame grabber to be the rtmp client works.

Does the AAC packets need to be decoded to PCM first to be usable ? Is there a way to do this without passing them to the framegrabber or can they be sent to a frame grabber using PipedInputStream ?

It's not a common use case it seems I only see microphone inputs that is PCM already. The server might have a utility already to decode the AAC packets to PCM to be consumable by the recorder.

Example of what I am trying to do right now but the output has to go to an outputstream for sending bytes to a websocket service.

FFmpegFrameRecorder recorder = new FFmpegFrameRecorder("D:\\test.ogg", 1);
            recorder.setFormat("ogg");
            recorder.setAudioCodec(AV_CODEC_ID_OPUS);
            recorder.setSampleFormat(AV_SAMPLE_FMT_FLT);
            recorder.setAudioOption("vbr", "constrained");
            recorder.setAudioBitrate(24000);
            recorder.setSampleRate(16000);
            recorder.setAudioChannels(1);   
 recorder.start();

Then in the packet capture method. I dont know the method yet to get the samplerate from the input packets. packet.getData is an AAC packet.

byte[] data = packet.getData();
ByteBuffer buffer = ByteBuffer.wrap(data);
recorder.recordSamples(48000, 2, buffer);

[saudet]

We can use grabPacket() and recordPacket() to copy them without reencoding, yes.

[danrossi]

Ok so I need to send the byte packet through framegrabber still via an inputstream ?

[saudet]

I'm not sure what you're talking about. You'll need to clarify your question to get an answer.

[danrossi]

Do I need to send the raw AAC bytes through a framegrabber like this

PipedInputStream pis = new PipedInputStream(); 
PipedOutputStream pos = new PipedOutputStream(); 
pos.connect(pis);

FFmpegFrameGrabber grabber = new FFmpegFrameGrabber(pis, 0);

Then in the packet capture

byte[] data = packet.getData();             
pos.write(data);                
AVPacket aac = grabber.grabPacket();                
recorder.recordPacket(aac);

[saudet]

No, we can do whatever we want! What do you want to do?

[danrossi]

Anytime I try and use Piped streams it hangs the process, that didnt work to be able to give framegrabber the bytes. The capture code was hanging from being called.

It needs to take in the raw AAC bytes from the RTMP stream packet capture in the server and convert to ogg/opus in the recorder. As per the original example. If I give it the bytes directly I get white noisy output in the test recording. I think the packet also includes headers not sure.

[saudet]

PipedOutputStream and PipedInputStream only work when we use them from different threads. You're not going to be able to use them from the same thread.

[danrossi]

So there is a workflow wrapping pcm from microwave into a buffer to record samples. Obviously piped stream isn't going to work. And so bytes to framegrabber isn't going to work. What is the same workflow for AAC packets ? If I give it the bytes directly I get white noise recorded. ie

ByteBuffer buffer = ByteBuffer.wrap(data)
this.recorder.recordSamples(48000, 2, buffer);

[saudet]

You probably have 16-bit samples, no?

[danrossi]

I'm sorry the AAC byte packets are direct from an RTMP encoder in FLV container I suppose. If I use ffmpeg as the rtmp frame grabber it works. But giving the recorder the bytes directly doesn't, I get white noise in the output. So I need to pass the bytes to the framegrabber to decode first somehow ?

[saudet]

I see, we can probably use FFmpegFrameRecorder.recordPacket() for that somehow.

[danrossi]

I've had to utilise decoded pcm data from the live streaming server for now. But what if I give it pre-transcoded opus packet bytes to wrap into ogg containers. Which is the requirement. Does the live byte packets still need to get into framegrabber to create an AVPacket ?