Integrating `ring-client-api`

[dgreif]

@bourdakos1 I figured it's probably easier if we pulled this conversation off on it's own. I appreciate your willingness to work together. I know how it goes when you get wrapped up in the technical side and do whatever it takes to just get it working.

Regarding your issues with getting a sip session through ring-client-api, I think I know what the issue is. By default, I do not poll the api for ding events. The homebridge-ring plugin turns this setting on by default, but I didn't want to force all users of the pure api client into polling. All you need to do is pass cameraDingsPollingSeconds: 1 (meaning poll every 1 second, you can change this to whatever value you prefer) to get active ding polling enabled. Then when you call createSipSession, it will hit the vod endpoint and the sip info will come down in the next poll.

Like I said in the other thread, you seem to know what you are doing with what you accomplished here. I could really use a second set of eyes to help figure out things like sip keep-alive and audio streaming to homebridge. Let me know if you are interested in helping out.

[bourdakos1]

@dgreif ahhhh okay, I didn't set cameraDingsPollingSeconds so that explains it. Does it make sense to automatically poll when calling createSipSession?

From what I've gathered, the SIP conversation goes a little something like this:

INVITE -->
<-- Trying
<-- Ringing
<-- OK
ACK -->
<==RTP starts==>

Your code sends 2 additional INFO requests (I'm not sure what they do) Nothing changes when I remove them 🤷‍♂️

In the initial INVITE is where we negotiate the initial terms of the call (the audio/video encoding formats etc...) You can change these terms by sending another INVITE within the same session and if the other party responds with OK, then the terms of the session are changed. I'm not sure if you can negotiate a change in the to field, if so maybe you can send an INVITE with a newly generate address.

I'm not sure if the standard keep-alive will work, just because the calls get terminated in clients like blink. I think with keep-alive you agree on a frequency and periodically send udp messages.

Also, calls are cancelled after 30 seconds if the other party doesn't receive the ACK, so that could be a silent issue

I'm not really familiar with homebridge so I don't know how much help I'll be with setting up audio. I can at least try and get audio outputted to a file like I did with video.

My main focus was just to get a video stream working, because I'm a computer vision nerd and I wanted to try detecting things like mailman/UPS or just some leaves blowing in the wind haha, but I'm glad to help get other things working as well :)

[bourdakos1]

Another thing is that whenever you start a dialog you get tags, in you code you don't keep track of the tag which can cause an issue in the dialog flow. When you send the INVITE the to param should come back with a tag, something like i3rye89qwoa, you need to keep that and send it with the next dialog message (in this case ACK) otherwise the other party might reject it because the tag wasn't specified

[bourdakos1]

Also what do you think about this for the SIP Session API? I can make some pull requests

const sipSession = await camera.createSipSession()

const h264Builder = new H264Builder('output.h264')
sipSession.videoStream.on('message', rtpPacket => {
  h264Builder.addPacket(rtpPacket)
})

sipSession.audioStream.on('message', rtpPacket => {
  // Do something with audio packet.
})

sipSession.on('end', () => {
  process.exit()
})

await sipSession.start()

if (forSomeReasonWeWantToStop) {
  sipSession.stop()
}

[dgreif]

All very good points! I'll try to respond to each:

• Automatically polling when creating a sip session is definitely a good idea. I can get that implemented this week
• The extra INFO dialogs are something that the android Ring app is doing. According to https://github.com/tchellomello/python-ring-doorbell/issues/23#issuecomment-326811495 it's required, though I think it works without them. Since they seemed optional and take time to execute, I split them out into their own function that gets called later in the flow for homebridge-ring. I wanted to get the main RTP info as quickly as possible because I need it to open the RTP ports locally. HomeKit is really strict in that in needs to know the SSRC for each stream before it will accept any packets, and I couldn't find a way to tell Ring what SSRC to use (tried X-SSRC-A and X-SSRC-V headers but no luck). In any case, we may be able to remove the INFO dialogs entirely.
• I didn't have a need to change the address with a follow-up INVITE, thought that might be a way for me to get the stream initially to get the SSRCs, then redirect the streams to the HomeKit hub...I'll consider giving that a try
• This is my first deep dive into SIP (I normally work with WebRTC) so a lot of the details were new. I didn't realize that tags were important, but I think my disregard for them is definitely part of the problem! I noticed that I receive a duplicate OK to the INVITE about every 10 seconds after I send the ACK. I think the lack of tag in the ACK is causing it to be rejected, which matches your logic on them terminating after 30 seconds. I would really appreciate a PR adding tags!
• The android Ring app sends an empty message (either '' or '\r\n', not sure which) through the sip socket every 30 seconds. I think this is how they are doing keep alive. I tried doing this by editing the sip module code, but it didn't work. I'm guessing my missing tags on the ACK were the reason it didn't work 🤦‍♂.
• I think the sip session interface can definitely be simplified and do more heavy lifting internally (like binding/opening ports and getting external IP automatically). The only change I would make to what you have above is that I really prefer using RxJS for all events. I know that node has been using the same on('event') model for forever, but I think RxJS is going to really start taking over in the years to come. It would be something more like sipSession.audioStream.onRtpPacket.subscribe(rtpPacket => {}). Thoughts? I completely agree with your start/stop methods.

Thanks for putting so much effort into this. It's really nice to have someone else with their head this deep into the code. I have a lot of users, but most of them are not developers and can't help solve some of these hard problems.

[dgreif]

I just added the tag to the to field and the stream now appears to stay open indefinitely. I think it needs to receive messages back through RTP/RTCP for it to stay open, but HomeKit generates that for me. Thanks for the info, I'll release an update right now!

[bourdakos1]

Oh that’s amazing! I did not think it would be that easy haha, how exciting!

I’m cool with reactive for the callbacks and I can try and get the RTP messages working for non homebridge support

And again I’m really sorry about not crediting you initially, you code was definitely key for solving this for me, I definitely just got way to excited when it finally started working

[dgreif]

No worries about the credit thing. I reacted impulsively because I had just put a ton of effort into that code last week, but you really didn't do anything wrong. I'm over it, and excited to have someone on board that I can bounce SIP issues around with!