iOS 10 HomeKit new Service Types

[leoneleone]

@KhaosT Will the new Services added to HomeKit (AirCon, Camera, etc) in iOS 10 be supported before the fall, for those of us using the iOS betas?

Perhaps in a separate fork or version on Github?

[KhaosT]

I won't be able to work on this project during the summer so if no one submits a PR, I can only do that in the fall.

Sent from my iPhone

On Jun 14, 2016, at 2:57 AM, leoneleone notifications@github.com wrote:

@KhaosT Will the new Services added to HomeKit (AirCon, Camera, etc) in iOS 10 be supported before the fall, for those of us using the iOS betas?

Perhaps in a separate fork or version on Github?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[microadam]

Started looking into this. @KhaosT I assume we need to wait until the HomeKit Accessory Simulator supports the new services? Or is there another way of getting their required data?

[legotheboss]

I'm very much interested in the camera, but I think we will need to wait until a newer version of HomeKit simulator is released.

[justme-1968]

the uuids and defined values should also be visible in the xcode framework files.

[microadam]

Possibly, although I couldn't find anything in the framework files by grepping for an existing UUID, so they aren't stored in place text amongst them

EDIT: I can actually see the UUID's in amongst some of the Binary files, but they are impossible to decipher any real meaning from

[bgottsch]

@microadam Yesterday, Apple released the updated version of HomeKit Accessory Simulator. Therefore, I was not able to find info on new accessories besides some audio characteristics (for the doorbell). Probably they will take some time to fully release all accessories, since we are still in the first week of beta 1.

Also, they released a video explaining the changes. I watched it, and there are some nice new features which I think will require minor changes to be implemented.

Since everything is covered by an NDA, it will be hard to discuss about the new features and changes publicly. For those with developer accounts, they updated HomeKit materials in the developer portal, and I believe a new video about a lab will be released today.

I believe a thread and an issue should be enough to get things going while @KhaosT is away.

[microadam]

New types (camera stuff) are available. Havent been able to figure out how to do anything with them yet though!

[smith288]

Home app doesnt even recognize the Camera accessory yet so currently can't find anything about it. Guess it's beta 3 that will support it?

[chrisboulton]

Home app doesnt even recognize the Camera accessory yet so currently can't find anything about it. Guess it's beta 3 that will support it?

It should - The HomeKit Accessory Simulator lets you add cameras (uses the FaceTime camera). For me as soon as I added a camera, I was able to configure it as an accessory in the Home app. It takes snapshots, and can stream live video/audio. I also regenerated HomeKitTypes.js to have a look at the differences, but haven't got any further than that as yet.

[smith288]

I had managed to create an IP Camera but the home app on iOS 10 b2 added it but said it wasn't supported. Did you get to that point yet?

[chrisboulton]

I had managed to create an IP Camera but the home app on iOS 10 b2 added it but said it wasn't supported. Did you get to that point yet?

Yup -

[smith288]

How did you set it up in accessory sim? I can't find a service that lets me choose a video

[KhaosT]

@smith288 You need macOS Sierra to see the option for video camera.

[smith288]

D'oh.... I should have known. Thanks...

[smith288]

Installing sierra now. Would be slick to have my garage door Pi Act as a bridge for all my IP cameras that I can integrate into HomeKit.

[KhaosT]

@smith288 Wait a moment, hap-nodejs itself doesn't support IP Camera yet. I mean HomeKit Accessory Simulator on macOS Sierra supports IP Camera...

[smith288]

No, yeah I know. Haha. Still want to play.

[leoneleone]

Have anyone made progress with adding the new Camera Service type to the HomeKitServiceTypes code?

[xmuskrat]

Excited to see future Camera support to HAP-NodeJS. Subscribing so I can keep up to date with any developments here.

[pdlove]

I've got a new HomeKitTypes.js file generated using KhaosT's original process. I'll upload it to my fork tonight. Do I need to wait for the final IOS 10 release to do a pull request? I'm sure there's more coding needed in hap-nodejs to support cameras after that, but it's at least the first step.

[leoneleone]

If your changes don't effect those users still on 9.3 I don't see why not?

Or those who'd like to experiment, like myself, can de directed to your fork 😉

[sembsa]

I want see cam on HAP-NodeJS :)

[KhaosT]

Meanwhile... do anyone know a good node.js library that supports RTP + SRTP for video streaming? If we can find one/make one then supporting IP Camera will be really easy.

[pdlove]

I've committed my changes to my fork. I don't know how long it will take, but I'm starting work tomorrow on a DLink camera insterface to use as a sample on how to use it. That being said, I'm only aware of how it works in very broad strokes. I will post more information tomorrow!

[leoneleone]

Any luck with the camera setup. I might modify my homebridge-nest plugin to work with the new characteristics. Never dealt with programming a camera before. Any advice would be much appreciated

[pdlove]

I've been having a bit of trouble. The problem with the camera is that it has to support SRTP to work natively. The HomeKit interface negotiates the connection between the user and the camera. I had a bit of help understanding the negotiation process, but the details are a bit more of a mystery to me. I'm trying to get an example negotiation from the HomeKit Simulator. If anyone wants to hire me a nanny and/or house cleaner it would free up some time to dissect this during the week. :) As @KhaosT mentioned earlier, we need a good node.js library for SRTP in order to make this work with almost any camera. Anyone have any thoughts or know anybody who could make one? I've also looked for SRTP to anything interface programs thinking that we may need to work towards another tool to handle the stream conversion outside of home bridge. I assume that once we've got this figured out that the stream will proxy through the iPad or Apple TV to be viewable from the Internet. That is just a hope on my part.

[leoneleone]

@pdlove I agree that some kind of SRTP interface would be the best way to go. While quite a few camera brands use SRTP, some popular ones like D-Link and Nest cams don't. I can't find a SRTP in node.js anywhere, so hopefully someone with more specific knowledge will chip in.

I wonder if having a look at how macOS sierra handles turning a laptop camera into a working homekit device...

Have you noticed any changes since the beta 6 release?

[leoneleone]

@pdlove I've just purchased a Foscam C1. It's a mini camera that supports RTSP, 2way audio, zoom, and motion detection. I'll try to see if I can get things working, although I'm not sure how to code the RTSP Configuration. I'll have a tinker and report back with news next week.

[el]

@pdlove I guess Gstreamer could be used to stream SRTP. There is a plugin: https://github.com/wernerd/gstzrtp If I am right, any camera (raspberry pi cam or webcams) could be used with software encoding and a little bit of delay.

[leoneleone]

Foscam has a plugin too. But it only currently functions as a HomeKit switch I believe, haven't tried it yet: https://github.com/rooi/homebridge-foscam

I think the FaceTime camera gets configured under a RTSP profile when being added to HomeKit. I understand that RTSP streaming is supported under macOS Sierra

Whichever way, we'd write up a new homebridge plugin to handle the new Camera Service.

I'm very pleased with the size and specs of the Foscam C1 and the Foscam C2. These are mini io cameras similar to the Nest Cam, maybe slightly bigger 🙂

[el]

@leoneleone How about this one: https://www.raspberrypi.org/forums/viewtopic.php?f=43&t=51476 I tried with VLC, it creates a RTSP stream.

[leoneleone]

@elizzk That's definitely a step in the right direction. I'm running homebridge on a pi. Perhaps writing a homebridge plugin which connects to the VLC RTSP you mention above. I'll have time to look into this over the week I hope. My ultimate aim is to find a way to get my Nest Cams working with the HomeKit Camera Service, but I'm happy to settle for adding new non-Nest camera which will do the job.

Does this new plugin you tried support a good and varies list of cameras?

Maybe @pdlove or @KhaosT could chip in with an opinion. They would be far more knowledgable than me.

[pdlove]

@leoneleone I'm learning as I go. At this point I think the two of you know more about RTSP than I do. I successfully sent and received an accessory request last night after pairing with HomeKit Simulator so I'm going to get my HAP accessory type for Homebridge thrown together and that should let me get some log grabs of the camera negotiation process. I've got some debug logs of this process, but I'm not sure how to make the TLV8 data packet that contains the information. More to come very soon! (Probably tomorrow)

[el]

@pdlove I used wireshark to see the packets but since they are encrypted I couldn't make any sense out of them. Do you know any other way?

[pdlove]

@elizzk I've reversed HAPServer to make a HAPClient which pairs and negotiates encryption with the accessory. This will eventually become homebridge-HAP which will allow you to connect home bridge into native HomeKit devices in order to consolidate the connections down. That also allows a data logging component to be added to home bridge that can be complete, but that's another topic entirely. At this time I'm pairing it directly to HAPServer to create a proxy that allows us to see cleartext. I'll also add the decoding of the TLV8 structure and output everything to text so you can "clearly" see the two way communication. Hopefully that will lay the secrets bare and we can finish cracking this mechanism. On another note, this also allows us to see plainly how real devices interact with the phone. So if we're having issues making something work reliably then we can hopefully find out why.

[leoneleone]

@pdlove @elizzk I'd just like to point out, regarding the Stay/Away (Arming/Disarming) feature which comes with most cameras: I think it best if this kind of "Arming" feature is integrated under: Service.SecurityService using the HomeKit Characteristics

/**
 * Characteristic "Security System Current State"
 */

Characteristic.SecuritySystemCurrentState = function() {
  Characteristic.call(this, 'Security System Current State', '00000066-0000-1000-8000-0026BB765291');
  this.setProps({
    format: Characteristic.Formats.UINT8,
    perms: [Characteristic.Perms.READ, Characteristic.Perms.NOTIFY]
  });
  this.value = this.getDefaultValue();
};

inherits(Characteristic.SecuritySystemCurrentState, Characteristic);

// The value property of SecuritySystemCurrentState must be one of the following:
Characteristic.SecuritySystemCurrentState.STAY_ARM = 0;
Characteristic.SecuritySystemCurrentState.AWAY_ARM = 1;
Characteristic.SecuritySystemCurrentState.NIGHT_ARM = 2;
Characteristic.SecuritySystemCurrentState.DISARMED = 3;
Characteristic.SecuritySystemCurrentState.ALARM_TRIGGERED = 4;

/**
 * Characteristic "Security System Target State"
 */

Characteristic.SecuritySystemTargetState = function() {
  Characteristic.call(this, 'Security System Target State', '00000067-0000-1000-8000-0026BB765291');
  this.setProps({
    format: Characteristic.Formats.UINT8,
    perms: [Characteristic.Perms.READ, Characteristic.Perms.WRITE, Characteristic.Perms.NOTIFY]
  });
  this.value = this.getDefaultValue();
};

inherits(Characteristic.SecuritySystemTargetState, Characteristic);

// The value property of SecuritySystemTargetState must be one of the following:
Characteristic.SecuritySystemTargetState.STAY_ARM = 0;
Characteristic.SecuritySystemTargetState.AWAY_ARM = 1;
Characteristic.SecuritySystemTargetState.NIGHT_ARM = 2;
Characteristic.SecuritySystemTargetState.DISARM = 3;

Any Motion Detection feature should obviously be linked to Characteristic.MotionDetected

This seems the most logical use for the Away feature, and I'm hoping it can be controlled using Siri Commands when its linked to the HomeKit Characteristics mentioned.

Please look at https://github.com/luisiam/homebridge-foscam2/blob/master/index.js for an example of this use.

I have been able to find my RTSP configuration for my Foscam camera now, but I'm very unsure how to format this into TLV8, so haven't tried modifying the homebridge-foscam2 code to try and add the new RTSPConfig Characteristics

@pdlove Any progress regarding TLV8 formatting?

[KhaosT]

Okay a little bit more information... As it turns out we need something that supports SRTP + RTCP Multiplexing. ffmpeg can only do SRTP but not multiplexing, vlc supports SRTP or RTCP Multiplexing but it appears you cannot use those two together... GStreamer + Funnel + SRTP might be feasible, need more evaluation on this.

[leoneleone]

I suspect there's more info regarding this which not available to general Apple developers, perhaps only just for those developers who are part of the MFI program. Looking through the HomeKit Documentation regarding cameras there's seem to be many details regarding Camera which is not elaborated on in the documentation.

[tmur11]

@KhaosT, @pdlove, @chrisboulton and @microadmin you are the kind of guys that inspire me. I'll be watching this thread very closely as this develops.

Any updates with the development?

@smith288 I was trying to run the homekit accessory simulator on macOS sierra but the camera service didn't appear avaiable, I downloaded the latest xcode beta and found the simulator in the developer tools section under "Hardware IO Tools for Xcode v7.3". I am running on a mac VM which currently doesnt have a camera connected but I thought i should at least get the device to appear in Homekit even if there is no video to display.

[todro]

@tmur11 I don't think this will work with the old Xcode V7.3 as the camera and doorbell device type were added recently.

I'm pretty sure you'll need to download the tools for a up-to-date Xcode V8 beta, the simulator for V8 beta 6 already works for me (Version 2.0 (108.2). While I couldn't manage to get the facetime cam to be routed to the VM guest, I ended in installing sierra on an external harddrive for testing until final release

[leoneleone]

@KhaosT

Now that iOS 10 has been officially released, when can we expect Hap-NodeJS (and homebridge) to be updated with the new changes to HomeKit? I can appreciate people are busy with other projects. Just wanted to know a timeframe, and whether this is being worked on.

@pdlove

Any updates regarding HomeKit Camera support. I'm still in the dark regarding how to make it work.

Thanks guys 🙂

[felixekman]

@pdlove and @leoneleone If HomeKit camera support can be implemented in an easier way I was thinking of making the cheap (Chinese version), but good YI Home cameras compatible with it and make a tutorial about it for others.

Yi Home Cameras http://www.yitechnology.com/homecamera/yut/id/6.html

[drott]

FWIW gstreamer's rtpbin seems to support RTPS using MIKEY key exchange.

[KhaosT]

With v0.4 release, I think most of iOS 10 HAP features are now available. For IP Camera, please refer the document https://github.com/KhaosT/HAP-NodeJS/wiki/IP-Camera

[Be2daErtin]

I've been trying this.... Can someone please assist on how to edit the camera.js file?

I use VLC to stream my http stream via rtp unicast to 239.255.12.42

I am able to view this on VLC for IOS. But HAP-NodeJS breaks:

events.js:141 throw er; // Unhandled 'error' event

my ffmpeg turns out to be (after removing all srtp):

-re -f avfoundation -r 29.970000 -i 0:0 -threads 0 -vcodec libx264 -an -pix_fmt yuv420p -r 30 -f rawvideo -tune zerolatency -vf scale=1280:720 -b:v 299k -bufsize 299k -payload_type 33 -ssrc 1 -f rtp rtp://192.168.2.8:60599?rtcpport=60599&localrtcpport=60599&pkt_size=1378

[KhaosT]

@Be2daErtin first you need to post the full error instead of just the line it says there is an error...

second, for ffmpeg, please don't touch the output part. Instead, what you should modify is the input section. -f avfoundation -r 29.970000 -i 0:0 <- this part. If your camera offers an rtp stream, you can try -re -i your_rtp_stream_address and the leave the rest of the parameters untouched.

[leoneleone]

@KhaosT Is there any way to get the Camera functionality to work with a URL address feed? I'm trying to find a way to get my Nest Cams integrated as HomeKit ip cameras. The only thing that Nest api exposes is a web_url which is a deep link to the camera feed from home.nest.com

Any thoughts on this would be much appreciated

[AdySan]

@leoneleone NestCams don’t provide a proper video feed do they? as far as I an tell, it’s a flash video, that offers scaling and stuff. Have you found a way to get the NestCam feed into something like say VLC?

[AdySan]

NestCams don’t provide a proper video feed do they? as far as I an tell, it’s a flash video, that offers scaling and stuff. Have you found a way to get the NestCam feed into something like say VLC?

Aditya

On Sep 27, 2016, at 2:50 AM, leoneleone notifications@github.com wrote:

@KhaosT https://github.com/KhaosT Is there any way to get the Camera functionality to work with a URL address feed? I'm trying to find a way to get my Nest Cams integrated as HomeKit ip cameras. The only thing that Nest api exposes is a web_url which is a deep link to the camera feed from home.nest.com

Any thoughts on this would be much appreciated

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/KhaosT/HAP-NodeJS/issues/253#issuecomment-249818965, or mute the thread https://github.com/notifications/unsubscribe-auth/AAtdakzGUAxGJ0CVoJo8a_WbK_w4vNGeks5quOcAgaJpZM4I1JqJ.

[leoneleone]

The don't provide a proper video feed as you put it, but the device.web_url deep link isn't limited to Flash, you can view the camera feed via the web_url for the Nest Cam on an iPad - it must must the feed using HTML5 as well