What if I want to send the PCMs to the server instead?

[binarykitchen]

Hello again

For my project www.videomail.io I am very keen to add sound. Wondering if your module could have another option to generate the WAV on the server side instead? I mean, there where you do the writeInt16LE(), send it to the server instead and write the WAV when end() has been fired.

This is probably a bigger request. Just sharing ideas for now.

[binarykitchen]

hello?

[ahdinosaur]

hey @binarykitchen. :wave: it's definitely possible to send the PCMs to the server, although i wonder if that's out of the scope of this module.

i'd be keen to see a new module (or a breaking change to this module) for writing WAV files from the output of read-audio, so you could use read-audio to get the audio data from a Web Audio input, then send that over the wire to the server, then the server could write the data as a WAV file. but that's just me.

cc @mmckegg

[binarykitchen]

oh hey kiwis

i really need an npm module badly where i can hook in a browser user media and have it send pcm files to the server, i.E. with websocket-stream, and encode these to a WAV file when done.

as a deaf programmer myself that's a bit difficult to hack. current experiments of mine do not work properly, see: https://github.com/binarykitchen/videomail-client/blob/master/src/util/audioRecorder.js https://github.com/binarykitchen/videomail-client/blob/master/src/util/items/audioSample.js

maybe i am messing up with the little/big endians in the audio sample? no idea ...

i think i am already in touch with @mmckegg regarding the above but he hasn't replied yet ... but if you have any idea, a hint, please let me know. i will definitely return the favour with beer donations or whatever you fancy. thanks heaps!

[mmckegg]

hey @binarykitchen !

@ahdinosaur can probably help you with this problem better than me, because it's not strictly Web Audio. It's more of a data format thing, which I'm not too experienced with.

But the biggest possible problem I can think of would be bandwidth. You would probably want to try and do the encoding in the browser if possible (but from a bit of research that sounds quite complicated).

Only other thing I can think of would be some kind of hack, where you attempt to stream the video and audio using WebRTC and record it on the server.

[binarykitchen]

thanks - in that case, @ahdinosaur, can i please, pretty please, ask for your service for very few hours?

@mmckegg few corrections:

bandwidth: worked on this for three years and there is no serious bandwidth issue with the video. don't worry about that. and it's always better to focus on functionality first and to optimize later ;)

webrtc: that's a no-go. videomail is like email, not synchronous and also works when the recipient is not online. whereas webrtc is a client-to-client solution only. studied webrtc for a long time and it's out of question.

[mmckegg]

@binarykitchen: I meant using WebRTC to the server not to another client, mainly so that you can take advantage of the built in encoding - you would then record the stream on the server then send it out as you do now. But it would probably be way more complicated than neccessary :no_good:

[binarykitchen]

@mmckegg i see - i studied that option before and webrtc doesn't communicate with the server nor has the functionality to send encoded stuff to the server. it works decentralized, so that wouldn't work for videomail.io

[ahdinosaur]

@binarykitchen your code seems fine, i'm not sure why it wouldn't work.

anyways, here's more on what i have in mind with my ndsamples modules.

// client.js
// get web audio input somehow
require('read-audio')({
  input: audioInput
})
.pipe(require('audio-coder-stream').encode())
.pipe(require('websocket-stream')())

// server.js
// ...
function handle(stream) {
  stream
  .pipe(require('audio-coder-stream').decode())
  .pipe(require('ndsamples-wav-stream').write('output.wav'))
})

keep in mind this is dream code: it depends on https://github.com/livejs/read-audio/issues/3, writing audio-coder-stream using audio-coder, and a mythical ndsamples-wav-stream module. these aren't on the top of my priority list (i'm building these modules in my free time to generate visual art from music), so my own progress will only happen as makes sense for me.

[binarykitchen]

hello @ahdinosaur

are you sure my code is fine? are you sure there is no mistake on how i treat the endianess on the client side?

and why the heck is the spectogram showing me no sound at all when i assemble all the pcm files to a WAV on the server? here's a gist how's done on the server side: https://gist.github.com/binarykitchen/f2a86ae2c4300a57381d

do you spot a mistake here?

a new npm package covering that dream code would be so awesome and exactly what i need here for videomail. i wouldn't mind to hack on this with you together, slowly over time.