search

d4rken-org / capod

A companion app for AirPods on Android.
https://play.google.com/store/apps/details?id=eu.darken.capod
GNU General Public License v3.0
491 stars 39 forks source link

AirPods Pro 2 in-ear detection not working well #38

Open Gallouche opened 1 year ago

Gallouche commented 1 year ago

Hey there, first thanks a lot for the app love it! Just bought AirPods Pro 2, and while setting up the app, I saw that the in-ear detection is not accurate, and it doesn't detect well if a pod is in-ear or not. While checking the monitor mode a bit and putting them on and off, I can see that the pod's status is either long to update or not correct at all. This is quite annoying for the auto-pause, which is not working, for instance. I don't have much more info to provide for now, but I would be glad to help debug!

d4rken commented 1 year ago

Is the detection wrong, does it display the wrong status in the dashboard even after waiting? Or is it just slow?

If the detection is wrong we need to look at how the data differs from AirPods Pro Gen 1.

If it's just slow you can try updating the monitoring interval in the settings.

If it's still slow there could be a change in behavior: Because we can't get the unique AirPod identifier (only Apple can), CAPod has to do some statistical analysis to determine which device is yours. If there are multiple supported headphones nearby, the detection can be a bit slower as the app may have lost your AirPods (doesn't know which one are yours).

The AirPods randomize their MAC address (every few minutes or when rebooted via case). If all other information is still the same, then CAPod can immediately recognize your AirPods. If unlucky and the AirPods have a new MAC address and change some data (e.g. "in-ear" status) in the same message, then CAPod will loose track and there is a delay until the AirPods are recognized again based on their signal level (how close they are). All quite complicated, but the best we can do for now.

Gallouche commented 1 year ago

Is the detection wrong, does it display the wrong status in the dashboard even after waiting? Or is it just slow? If the detection is wrong we need to look at how the data differs from AirPods Pro Gen 1.

It's a bit of a mix of both. For instance, I looked at the monitor when removing one airpod, and the behaviour was: pull out the airpod => no change detected, even after a long time => put the airpod in case => change detected (quite slow too if I remember). This makes me think that detection could be wrong... Would be happy to try more scenarios if you need them.

If it's just slow you can try updating the monitoring interval in the settings.

Yes already put the setting on low latency

d4rken commented 1 year ago

Does it work better if you use a slower latency mode?

d4rken commented 1 year ago

pull out the airpod => no change detected, even after a long time => put the airpod in case => change detected (quite slow too if I remember). This makes me think that detection could be wrong...

That means that the issue isn't in the play/pause code. Which is good but then it's also not as easy a fix.

The case is a bit of a special "case". If you add/remove a pod from your ear (or just cup with your hand), does it always not work? What if you remove one AirPod (nothing changes?) then remove the second AirPod, does it then change?

To rule out a decoding issue, you can enable debug mode and the on screen card should display the raw data in hex values. Even if the UI does not update the in-ear state, does the raw data change?

Edit: In debug mode, does the receive counter in the top right corner keep going up? Maybe this is after all related to CAPod loosing track of your specific AirPods.

Are any other around? I wonder if we have code yet to take a shortcut if there is only ONE pair of AirPods around, or if there are multiple but only one of the type the user selected :thinking:

Gallouche commented 1 year ago

Does it work better if you use a slower latency mode?

Nope

The case is a bit of a special "case". If you add/remove a pod from your ear (or just cup with your hand), does it always not work? What if you remove one AirPod (nothing changes?) then remove the second AirPod, does it then change?

It's a bit complicated to be very precise because the behaviour seems a bit messy (looks like the app misinterpret data to me). Following your scenario:

Additional things I noticed:

does the receive counter in the top right corner keep going up?

From what I saw during the described scenario before is that the counter is not resetting and just going up. But I can also see that sometime it reset kinda randomly (need more observation, maybe related to case opened or not)

Are any other around?

No, not in my apartment, at least

This sounds like a decoding issue indeed, I may have a look this weekend at the raw data, learning to sniff packets from BLE soon!

Gallouche commented 1 year ago

Also, one more question to better understand, Is the debug logger logging every raw data received from the device or only the one it knows / can decode?

d4rken commented 1 year ago
  • Both AirPods in case = > both in charging mode, left have mic enable?

Not so much "left mic enabled" but "left mic priority", left is the default, afaik.

  • Put right one => right in-ear, micro enabled on right
  • Put left one => left and right in-ear, micro enabled on right
  • remove left => nothing changed, not even the debug log
  • put back left => nothing changed, not even the debug log
  • remove right => left in-ear + mic enable, right not in-ear anymore
  • put back right => left in-ear + mic enable, right not in-ear, debug log didn't changed

If the debug data did NOT change, then it can't be a decoding issue though? We can only decode what is received :shrug:

  • when both pods are in the ear, the case status is N/A, and open-close is not detected

For gen1 the case does not have a Bluetooth, it only communicates through the pods, i.e. when one is in the case. I think the gen2 case also doesn't have a Bluetooth chip.

  • when I cover my ear wearing an airpod with hand, the case status switch between close and open

That sounds like a decoding issue.

AirPods 2 are defined here but they mostly inherit from the base "dual pod" class here

I may have a look this weekend at the raw data, learning to sniff packets from BLE soon! Also, one more question to better understand, Is the debug logger logging every raw data received from the device or only the one it knows / can decode?

The raw data shown in the UI in debug mode is just the raw data that is then also shown by the icons in decoded form. Unknown decoded data that still looks like an Apple device would show as "unknown apple device". If you also turn on "unfiltered BLE data" in settings, then data from all nearby BLE devices is shown even if completely unsupported.

If you use the "record debug log" option it will contain all BLE advertising packets the phone receives. Working with log files is a bit time consuming though. For playing around with this the easiest would probably be to just install Android Studio, and open this project in it, and then just install the dev version to watch all of it in the log viewer.

d4rken commented 1 year ago

Here are some more hints for your research:

Android calls what is received ScanRecord, looks like this (visible in debug logs or in dev builds in the log viewer):

ScanResult{
    device=64:62:BE:02:A7:58,
    scanRecord=ScanRecord [
        mAdvertiseFlags=-1,
        mServiceUuids=null,
        mServiceSolicitationUuids=[],
        mManufacturerSpecificData={
            76=[7, 25, 1, 14, 32, 85, -86, -71, 49, 0, 4, -80, -19, -12, -77, -40, 106, 17, -114, 117, 106, 96, -16, 118, -35, -28, -126]
        },
        mServiceData={},
        mTxPowerLevel=-2147483648,
        mDeviceName=null
    ],
    rssi=-52,
    timestampNanos=1442702574300997,
    eventType=17,
    primaryPhy=1,
    secondaryPhy=0,
    advertisingSid=255,
    txPower=127,
    periodicAdvertisingInterval=0
}

mManufacturerSpecificData is what contains our raw data. 76 is the identifier for Apple. 7 is the identifier for Apple continuity protocol and 25 the payload length etc. For the rest, checkout the paper I linked in the README.md or view a few of the unit tests.

Gallouche commented 1 year ago

Thanks a lot 🙏🏼 I'll try to play a bit during the weekend and keep you up to date.

Gallouche commented 1 year ago

I tried to check a bit, but very newbie on this 😅 From what I see in logs when removing an airpod, the data are not updated, while the counter is still going up (so AirPods sending the same data over and over? ). So as you said, maybe it's not a decoding issue 🤔 Strange as this works perfectly on my macbook (pause when removing one pod). This is very inconsistent because one time, a bit randomly after removing one from the ear, I saw the change like 20- 30 sec after even in low-latency mode (was able to see the same data sent in the meantime in Android Studio debugger). Cannot help much more for now. Maybe I'll see if I can dig deeper, but I'm very new to Kotlin and BLE dev, so it's very slow to understand what is happening in the code 😄 Maybe you would be interested in some more logs? (would try to limit the unwanted devices in the log by going out of the building or else)

d4rken commented 1 year ago

Maybe you would be interested in some more logs? (would try to limit the unwanted devices in the log by going out of the building or else)

If you see the same data being received, then I wouldn't really know how what to look for. Is the data in the logs really 100% the same? On Apple device the data updates immediately? Can you view the status on Apple devices without the pods being paired? There are two parts to the broadcast data, unencrypted part (that we use), encrypted part (only Apple can use). But then there should be a visible change in the data, at least the encrypted part.

d4rken commented 1 year ago

See also: https://github.com/steam3d/MagicPods-Windows/issues/246

steam3d commented 1 year ago

Do not lose time. There is no way to detect ear on 5B58 firmware. We must wait a new firmware version maybe something will change.

I did a lot of test with any settings of AirPods Pro 2.

For @Gallouche who said that AirPods Pro 2 good works on mac os etc it is because apple use other protocol to detect ear position that much faster than BLE.

mainrs commented 1 year ago

For @Gallouche who said that AirPods Pro 2 good works on mac os etc it is because apple use other protocol to detect ear position that much faster than BLE.

As far as I know, the Airpods also send data that has yet not been properly decoded. Is it known for sure that the data is encrypted? If so, is that encryption Apple specific, or can it be analyzed and reversed as part of some kind of handshake?

Is the ear detection information possibly send in that undecoded data blob?

steam3d commented 1 year ago

@mainrs I know that part of ble data encrypted. It doesn’t contain ear detection data.

mainey commented 1 year ago

Just to let you know, the newer airpods pro 2 fw 5E133 still doesn't fix things (at least for me).

salmannotkhan commented 1 year ago

@d4rken I just noticed that in ear detection doesn't work at all when playing music. But as soon as I pause music it starts working.

steam3d commented 1 year ago

Airpods pro 2 does not support ear detection

salmannotkhan commented 1 year ago

It does work when music isn't playing tho 🤔

mainrs commented 1 year ago

It does work when music isn't playing tho thinking

What do you mean exactly? In-ear detection is usually used to automatically pause and resume audio playback when you take them out of your ear.

Do you mean that, if the music is paused and you put them into your ear, the music resumes playback automatically?

salmannotkhan commented 1 year ago

Yes exactly, If I pause music and wear the airpods it resumes the music.

mainrs commented 1 year ago

Yes exactly, If I pause music and wear the airpods it resumes the music.

At least I have never encountered this behavior. What device do you use? What firmware is on your airpods pro v2?

salmannotkhan commented 1 year ago

I am using Poco X3 (with LineageOS 20) Airpods Firmware: 5E135

mainrs commented 1 year ago

I run GrapheneOS on a Google Pixel 6. I can confirm that with firmware 5E135 autoplay works when putting the earphone back into your ear. It does not work when removing it though.

However, it is really spotty. It does not work all the time and it worked only with the left earbud for me. I could not reproduce it with the right one.

d4rken commented 1 year ago

Don't fixate too much on whether music playback starts or ends. This is only a reaction to conditions. The important part is whether the "in-ear" detection status changes correctly when looking at the data (e.g. in CAPod).

salmannotkhan commented 1 year ago

@d4rken It does change correctly when music isn't playing

salmannotkhan commented 1 year ago

I will clone the repo tonight and tinker around the in-ear detection. Let's see if I can figure it out

steam3d commented 1 year ago

You could not spend time on this. I made planty of tests and can confirm that ear detection will not work on none Apple devices. Apple detects when you listen to music on none Apple device and stopping sending data.

kavishdevar commented 3 months ago

any updates on this issue? also, i've noticed that when i'm using the airpods with my mac, the app detects if they are in ear almost immediately, but not when connected to the phone itself.

kavishdevar commented 3 months ago

here's another thing i noticed, the app detects the airpods if they are removed way quicker and accurately if no music is playing, but take a long time to detect if something is playing. and as mentioned by others, resuming works normally and quickly. Edit: seems like the detection of putting the airpods on is also only detected if music is not playing, if i am listenting to music with just one airpod, and i put the other airpod, the status in the app isn't updated...