opawg / user-agents

An open, platform-agnostic list of user-agent and referrer regexes for use in podcast analytics services
MIT License
122 stars 71 forks source link

Incorrect user agent for Apple Podcasts #119

Closed joshf closed 1 year ago

joshf commented 2 years ago

I noticed that for Apple Podcasts on macOS Monterrey that the following user agent is used:

AppleCoreMedia/1.0.0.21G83 (Macintosh; U; Intel Mac OS X 12_5_1; en_gb)

In the list it states that AppleCoreMedia should not be treated as Apple Podcasts but in this case it should. I am not at which point Apple changed the UA but it means that platform detection does not work as expected

Thanks!

liamwhiteside commented 2 years ago

@joshf I think the issue is that your example UA could be Apple Podcasts, but it could be something else using that Apple library (and in many cases, especially with iOS apps it has been). So we can't say for sure that everything with that UA is someone using an Apple Podcasts app. I think the Apple apps still use a different UA depending on whether you're downloading as you're listening, or downloading to a file in the background. The second of those is normally when AppleCoreMedia is used, as the download is handed off to a background process.

joshf commented 2 years ago

more info on which user agent is which:

clicking play:

AppleCoreMedia/1.0.0.21G83 (Macintosh; U; Intel Mac OS X 12_5_1; en_gb)

clicking download: Podcasts/1660.5 CFNetwork/1335.0.3 Darwin/21.6.0

johnspurlock commented 2 years ago

I think the Apple apps still use a different UA depending on whether you're downloading as you're listening, or downloading to a file in the background. The second of those is normally when AppleCoreMedia is used, as the download is handed off to a background process.

It's actually the opposite. Apple Podcasts (both on mac and mobile) and all other iOS apps, send a proper UA when downloading, since they can completely control the http request. They all use another framework (AVKit/CoreFoundation) when streaming, which makes the requests for them, using AppleCoreMedia as the UA. (Overcast doesn't stream at all, so never sends AppleCoreMedia.)

Prior to iOS 16, which was just released, Apple provided no way for developers to customize the User-Agent in the streaming case.

Good news is that there is now a way to do it! Bad news is that none of the podcast apps (including Apple's!) use it yet.

James has a great page summarizing this here.

joshf commented 2 years ago

so for now its just a case of hoping apple updates it's app to use the new UA?

johnspurlock commented 2 years ago

so for now its just a case of hoping apple updates it's app to use the new UA?

Yep, and the other iOS/macOS apps. Future is bright since we have a constant [1] to use now, just a matter of folks getting around to using it. Fortunately Apple platforms update to latest OS far more quickly than Android, so it should be a matter of months, few years max

[1] https://github.com/johnspurlock/swift-playgrounds/blob/master/avplayer-override-user-agent-16.swift#L48

jamescridland commented 1 year ago

Thanks for those here. AppleCoreMedia should be detected by the OPAWG list, but won't resolve to an app name, since it isn't an app, just an internal SDK.

For October 6, here are the apps that used AppleCoreMedia as a UA for the Podnews podcast:

Screen Shot 2022-10-08 at 9 16 53 am

Is this good to close for now?

joshf commented 1 year ago

Yeah thank you all for the clarification, let's hope people start to use the new sdk stuff soon!