search

InfiniTimeOrg / InfiniTime

Firmware for Pinetime smartwatch written in C++ and based on FreeRTOS
GNU General Public License v3.0
2.74k stars 935 forks source link

Implement Apple Notification Center Service #910

Open ryangodburn opened 2 years ago

ryangodburn commented 2 years ago

Verification

Pitch us your idea!

Add the Apple Notification Center Service to allow for notification access on iOS.

Description

What is blocking Infinilink on iOS from notifications working is that the Apple Notification Center Service is not implemented, so if it can be added then notifications could work from iOS.

JF002 commented 2 years ago

Duplicate of # 460

ryangodburn commented 2 years ago

Apologies thought this would be a different thread for the notification service, not the music service.

JF002 commented 2 years ago

Oh sorry, my bad! You're right! We talked about music control yesterday, and I mix them up with notifications! I'm removing the duplicate!

Sooo... Apple also need a specific API for notifications?

ryangodburn commented 2 years ago

Unfortunately, it is so. I have found some more information about it here. I would be more than happy to test if anything could be added.

TheAwesome98-Real commented 2 years ago

You should probably move this issue to InfiniLink since that’s where it should be implemented

ryangodburn commented 2 years ago

The language used on its GitHub makes it seem like it needs to be implemented into the Infinitime firmware itself. image

TheAwesome98-Real commented 2 years ago

Hm, that's weird. I thought all InfiniTime does is send play/pause/skip etc. requests to the device and the companion app (InfiniLink in this case) handles it.

JF002 commented 2 years ago

@TheAwesome98-Real My understanding is that iOS notifications and music control can only happen on a specific Apple branded BLE API... So InfiniLink has no choice but use the Apple protocol instead of the more generic and standardized protocols currently supported by InfinITime.

xan-m commented 2 years ago

InfiniLink dev here! @JF002 and @Ryanmp4 are correct, notifications require ANCS (Apple Notification Center Service) and music requires AMS (Apple Music Service). I can't do anything from the iOS side to force notifications from outside InfiniLink through the existing notification API on the watch. InfiniTime does send play/pause/etc. to iOS, but without AMS I only can use InfiniTime's media characteristic to control Apple Music (not Spotify, Netflix, podcast apps, etc).

I've tried a couple of times to implement ANCS in InfiniTime, but unfortunately I can't really wrap my head around the level of C++ in InfiniTime... The link the @Ryanmp4 listed above is the complete documentation to implement the notification service, so it's just a matter of someone with the ability and bandwidth to hack it in!

For reference, AMS spec is here.

ck-telecom commented 2 years ago

I have tried / ANCS / and AMS on my Zephyr pinetime project https://github.com/ck-telecom/pinetime, and I can recieve coressponding notification and entity attribute from console, but I have not make it on my UI.

minacode commented 2 years ago

If I would try to implement this feature: how safe is it to test this on a sealed watch? Otherwise I could still try to contribute code (or finally have to order a devkit 😄).

JF002 commented 2 years ago

It should be quite safe as you'll add new features. You'll probably not touch the low-level driver or startup code so you'll be pretty safe. In case of issue, the bootloader should always be able to revert to the previous version and load the recovery firmware as a last resort.

scandinave commented 2 years ago

@minacode I already start to implement ANCS. maybe we can work on it together. i can add you to my repo as contributor.

minacode commented 2 years ago

Sure, I would like that 😊

scandinave commented 2 years ago

Sure, I would like that 😊

done

ryanfortner commented 2 years ago

Hi there, has there been any progress on adding the notification service?

minacode commented 2 years ago

For me, not much, sorry. I do not have much time right now, but I tried to implement ANCS for Linux to learn how it generally works. This is still in progress, because

Due to the nature of iOS, the ANCS is not guaranteed to always be present. As a result, the NC should look for and subscribe to the Service Changed characteristic of the GATT service in order to monitor for the potential publishing and unpublishing of the ANCS at any time.

This is not fun to implement for me even on linux, so unless I am confident with this on a well debuggable device, I will not touch this on the Pinetime. The plan ist not dead though. I would still like to implement this in the hopefully-not-so-far future.

ryanfortner commented 2 years ago

I understand. Thanks for the update.

jrmolin commented 2 years ago

I have a dev kit on the way. I would also like to get some notifications on my iOS device. Do you want to collaborate? I have a build environment setup, and I already modified one watch face (analog to get step counts on there) and OTA flashed it. No problems so far! However, I didn't want to OTA-only flash anything mucking with the guts of notifications.

minacode commented 2 years ago

The progress already made is here.

I can do a small writeup of my last progress there soon.

Also, i found OTA on the sealed watch to be safe, as long as you don't verify the new image, because you can always reboot into the previous firmware via holding the button pressed. At one point I debugged generating notifications by showing a new one at each touch input. It worked, but made the watch pretty much unusable 😄 The same should hold when messing with Bluetooth, where you might lock yourself out from OTA (I guess?) but can still just go back to the previous firmware. Despite trying a lot of builds I never got to this point.

MagicTrevor commented 2 years ago

Any more progress? I am getting a mini soon and want to start helping around this as well.

minacode commented 2 years ago

No, no progress sadly. I wrote a comment on the draft.

There are some architectural hurdles, because right now, InfiniTime seems to initialize all Bluetooth services upon connection. This is fine, because they are just always there. But the ANCS can appear and disappear at any moment. I made a test implementation of ANCS on my laptop and it showed, that the ANCS is normally not available directly after connecting to my iPhone.

So the question is: How can we build the ANCS such that it can be on and off at any moment, determined by the GATT services ServiceChanged characteristic?

Edit: A good first step would be to implement the GATT service (0x1801) and its ServiceChanged characteristic.

cole-wilson commented 2 years ago

Where is the status of this now? Is there a proof of concept available? I would love to help out any way I can, but I don't think I have the necessary c++ skills!

minacode commented 2 years ago

No progress. What kind of proof of concept do you mean? I can run ANCS on Bluez/Linux. And the watch can connect to devices via BLE. ANCS on the watch is not implemented. We should start by handling the GattService, as I mentioned before. Sadly I don't see myself contributing this until September, because I just don't find the time.

MagicTrevor commented 2 years ago

Isn't an encrypted connection required first? Or has that issue with iOS been handled?

minacode commented 2 years ago

I could never do it, but someone reported to have done it by encrypting the battery service, I think. I am not 100% sure though.

Edit: #920

Just to reiterate: when testing a build from early in the bonding PR process (when the battery level characteristic was used to trigger bonding), the instant I connected the PineTime the bonding prompt with the PIN showed up on my phone and the watch. I don't have a sniffer so I'm not 100% sure to what degree my communications were encrypted after that bonding, but if my understanding of the process is right, it should only have to be one characteristic that requires authentication.

My hope is, that one of the Apple services is sufficient to trigger encryption.

shymega commented 3 months ago

I see there's a PR here, so I've pulled it into my fork and rebased it against main.

However, I think it would be best for the PR to be opened on this repo for collaboration. The PR on the fork was hard to find.

I've had experience with BLE characteristics with earbuds, so I'm fairly okay working with Apple's service here.

I'll keep checking in here as progress moves. I have an iPhone 13.

minacode commented 3 months ago

Does it work? I would be very surprised.

shymega commented 3 months ago

It doesn't yet. I'm reading through the Apple spec when I have time. I have a devkit PineTime, so I can debug. Just waiting for my probe-rs device though.

minacode commented 3 months ago

Nice! Debugging is what made me quit, because it wasn't really possible between a sealed watch and an iPhone. I would love to contribute if you get something going, though. If you have questions, feel free to ask. I wrote an ANCS implementation that ran on Linux when I worked on the PR.