Audio not supported

[Jeremy1026]

Audio appears to no be supported with the video feeds. Is there a way, or plan, to add support?

[KhaosT]

Yeah audio is not supported. Not really sure I’m going to support it, as ffmpeg itself isn’t really flexible on handling this…

Sent from my iPhone

On Mar 13, 2018, at 7:05 PM, Jeremy Curcio notifications@github.com wrote:

Audio appears to no be supported with the video feeds. Is there a way, or plan, to add support?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[leoneleone]

@KhaosT Thank you for the plugin, its very useful. I'd like to put in my vote for audio support. With the new Nest Hello (doorbell), it seems imperative that audio support be added, otherwise the doorbell would have very limited compatibility in HomeKit.

[leoneleone]

@KhaosT Thanks for continuing to make this plugin better with each update.

I really believe that 2 way audio support essential for the Nest Hello doorbell cam. Are there any methods we can test to get 2 way audio on the Nest Hello in particular?

I already use your plugin on a Nest Hello, in conjuction with homebridge-nest (motion sensor) and get ok results with it. Once Nest release specific api for the Nest Hello (Button/Doorbell press event), we can create a Doorbell Service with the relevant Characteristic.ProgrammableSwitchEvent.SINGLE_PRESS for the Nest Hello in homebridge-nest based on the future “Doorbell press event”.

Audio Support is completely integral for the Nest Hello to function as a useful HomeKit Doorbell Camera.

Any thoughts on this?

[kitknox]

I did start looking into adding audio support. The current NexusStream handling is generating a play request which is actually receiving the audio. I dumped the raw OPUS packets and then converted them offline into PCM and was able to confirm playback. In terms of getting this data into the FFmpeg pipeline the first problem is that FFmpeg doesn't support OPUS packets without them being inside another container. I think the easiest thing would be to direct decode to PCM with libopus and then shove the PCM data back into FFMPEG for AAC conversion which the iOS ecosystem is going to be happier with.

Adding support for two-way audio would involve further Nest protocol support. It looks like some of the hostnames for this are already coming back in JSON responses. Not sure if that is more of the protobuf stuff like what is used for playback.

[ChrisKader]

@kitknox do the nest cams work with WWGA since WWN is no longer letting people create stuff? I would love to update this library to support the merged google accounts and along with audio but I am having trouble finding a "Supported Devices" section.

[rollinnerdy]

I would like to take a stab at this but I am struggling to find where the specification for the nest camera stream came from. Can you provide links the stream spec or where you got the proto file to generate the protocol sources.