search

salomvary / fdk-aac.js

Port of Fraunhofer FDK AAC with Emscripten
27 stars 7 forks source link

Use in real time #2

Open ROBERT-MCDOWELL opened 2 years ago

ROBERT-MCDOWELL commented 2 years ago

Is frame by frame or in a loop to encode raw audio into aac possible? If yes I would love to see an example

salomvary commented 2 years ago

I don't know, I suggest you check out the original Fraunhofer FDK AAC library and see what it is capable of. Likely adjustments will be necessary in this repo as well.

ROBERT-MCDOWELL commented 2 years ago

ok thanks

tchakabam commented 2 years ago

@ROBERT-MCDOWELL probably yes, audio encoder libs operate at that level of granularity. in fact, there isn't much other way, the input data can only really by framed by raw audio samples.

that is, to create one AAC frame you need 1024 audio samples, so that will be the min input buffer size.

ROBERT-MCDOWELL commented 2 years ago

@tchakabam thanks! if you know some simple example of how to smatrly manage the buffer and how to stream it it would be nice.

tchakabam commented 2 years ago

Hi @ROBERT-MCDOWELL , sorry missed this message. You can email me at stephan@emliri.com

Your question is pretty broad, I am not sure what you really want to know at this point :)

ROBERT-MCDOWELL commented 2 years ago

ok thanks will do @tchakabam I just wonder if it can be integrated into webcodecs API.

tchakabam commented 2 years ago

Webcodecs should already give you an encoder (if the respective browser has it). so you wouldnt need this library.

Or do you mean Webaudio?

ROBERT-MCDOWELL commented 2 years ago

mhmm, I thought webaudio was included in webcodecs.... very confusing... so yes webaudio. I recently talked to a dev guys from webcodecs to integrate custom audio/video encoder decoder as plugin into the stack and he said not for now but nothing is closed about it.

ROBERT-MCDOWELL commented 2 years ago

here is the discussion https://github.com/w3c/webcodecs/issues/413

tchakabam commented 2 years ago

mhmm, I thought webaudio was included in webcodecs....

no :) those 2 are really 2 completely different stories. and there isn't any ambiguity.

webaudio is to use audio devices as in sound output, like your speakers / soundcard, or get input from microphone. (there is a lot of docs on webaudio btw, so you can just read up whatever you want to know)

webcodecs is to use a/v codecs that your platform offers, but that can be unrelated to wether you will use a soundcard.

hope that's helpful.

tchakabam commented 2 years ago

I recently talked to a dev guys from webcodecs to integrate custom audio/video encoder decoder as plugin into the stack and he said not for now but nothing is closed about it.

i don't think that is really the point for webcodecs. it is mainly about offering direct access to (possibly hardware accelerated) a/v codecs that are usually platfrom native owned.

i don't see what wrapping any custom codec impl into the webcodecs api would solve itfp here.

ROBERT-MCDOWELL commented 2 years ago

also the biggest problem is mon threading limiting drastically the web browser. is webaudio open the door to integrate new audio codecs?

ROBERT-MCDOWELL commented 2 years ago

sorry for my question but I still do not see the difference between webcodecs and webaudio, unless webcodecs is for video and webaudio for audio?

ROBERT-MCDOWELL commented 2 years ago

oops I just read your answers.... ok so let's imagine an exemple, using webaudio for audio inputs and outputs, grab the raw audio/video and use webcodecs to encode in my custom codecs, attach the frames to a net stream, send it, and play it from raw data by decoding the stream with the wbcodec of my choice. This is I would like to do, removing the webrtc gas machine which the developer cannot custom anything in it.