mazjap / NeoMusic-SwiftUI

10 stars 1 forks source link

is this still being maintained? #2

Closed MichaelJoo closed 1 week ago

MichaelJoo commented 1 week ago

Does this support "NowPlayingView" for both Apple Music and Spotify?

mazjap commented 1 week ago

This project isn't actively being maintained, but I would like to get back into it. Perhaps this issue will serve as a reminder/motivation for me :)

Currently, there is no preliminary support for Spotify. It is a feature on my TODO list, although I haven't looked into the details of it (whether Spotify has a publicly available API for searching/streaming music, whether said API has a Swift interface, etc.) There is support for on-device Apple Music songs.

That said, this project uses the system-level MPMusicPlayerController (as apposed to an application-level MPMusicPlayerController), meaning any audio already playing on the device will be visible to NeoMusic: Spotify, Soundcloud, Podcast, Youtube, etc. There just isn't support for starting or searching for audio from those sources. I was incorrect here, a possible solution has been provided below.

Apple has come out with a new API for music playing, MusicKit, which has a lot of built-in quality of life features that I would love to implement into this project. Perhaps while doing that, I could incorporate a Spotify API as well (maybe other music streaming services as well).

What is your use-case for NeoMusic, if I may ask?

MichaelJoo commented 1 week ago

I found your repo while trying to build a system level "NowPlayingView" that can possibly display music or media from all sources such as Apple Music, Spotify, YouTube Music, etc.

However, at this point, I have only built NowPlayingView for Apple Music (just 1 view controller file, much simpler than NeoMusic), so I am just trying to find any repo that has already done it.

That said, this project uses the system-level MPMusicPlayerController (as apposed to an application-level MPMusicPlayerController), meaning any audio already playing on the device will be visible to NeoMusic: Spotify, Soundcloud, Podcast, Youtube, etc. There just isn't support for starting or searching for audio from those sources.

This is what exactly I am looking for

    private var player: MPMusicPlayerController = .systemMusicPlayer {
        didSet {
            dynamicPlayer = Dynamic(player)
        }
    }

I am also using private let musicPlayer = MPMusicPlayerController.systemMusicPlayer but I was not successful in displaying info from podcasts, youtube music etc by using let nowPlayingInfo = MPNowPlayingInfoCenter.default().nowPlayingInfo Right now, NeoMusic doesn't succeed in building so I am just trying to read, and see if it is even possible to do what you mentioned, to get info from podcast, not only apple music.

MichaelJoo commented 1 week ago

Just FYI, most "Now Playing Views" in Apple apps are supporting Apple Music and Spotify. And Spotify apparently has iOS SDK that can enable such NowPlayingView, although I haven't got into the details.

Flutter has similar repo that can support Apple Music and Spotify, so I assume it can be done. But the only thing that I am trying to figure out, is whether can "podcasts" or 'youtube music" information can be shown in this "NowPlayingView".

I am building this "NowPlayingView" as an addition to my workout app.

mazjap commented 1 week ago

I was mistaken. Apple is very privacy-focused, and allowing system-level MPMusicPlayerController to see media from other apps, aside from Apple Music, could be a privacy concern (e.g. if an app stored data about every video you played/audio you listened to). I have a solution, though, albeit hacky.

Note, this is using Apple's private APIs and thus will not be accepted onto the AppStore. But if your use-case is strictly for personal/learning use than this will work, with the caveat that private APIs can change in any future iOS release, possibly making this code not work anymore.

Here's the original source I found and then adapted

Steps:

  1. Add Dynamic as a package dependency to your project via Swift Package Manager.

Dynamic is a package that allows you to access private Objective-C types, properties, functions, and even frameworks from Swift and is necessary to achieve access to MRMediaRemote which will be used to get now playing info from other apps. It's a package I use in NeoMusic to get access to the private nowPlayingItemAt(index:) method on MPMusicPlayerController for NeoMusic's queue as well as a couple other things.

I used this library of private frameworks in the iOS SDK (and more specifically this file) as documentation for what symbols to try out.

  1. Add this code to your project and modify as needed:
import MediaPlayer

/// Use private MediaRemote framework to get system wide now playing info
func fetchPrivateApiData() {
  // Load private MediaRemote framework
  let bundle = CFBundleCreate(kCFAllocatorDefault, NSURL(fileURLWithPath: "/System/Library/PrivateFrameworks/MediaRemote.framework"))

  // Get a Swift function for MRMediaRemoteGetNowPlayingInfo
  guard let MRMediaRemoteGetNowPlayingInfoPointer = CFBundleGetFunctionPointerForName(bundle, "MRMediaRemoteGetNowPlayingInfo" as CFString) else { return }
  typealias MRMediaRemoteGetNowPlayingInfoFunction = @convention(c) (DispatchQueue, @escaping ([String : Any]) -> Void) -> Void
  let MRMediaRemoteGetNowPlayingInfo = unsafeBitCast(MRMediaRemoteGetNowPlayingInfoPointer, to: MRMediaRemoteGetNowPlayingInfoFunction.self)

  MRMediaRemoteGetNowPlayingInfo(DispatchQueue.main) { information in
      print(information) // To get keys and values
      let itemId = information["kMRMediaRemoteNowPlayingInfoContentItemIdentifier"] as? String
      let artistName = information["kMRMediaRemoteNowPlayingInfoArtist"] as? String
      let songName = information["kMRMediaRemoteNowPlayingInfoTitle"] as? String
      let albumName = information["kMRMediaRemoteNowPlayingInfoAlbum"] as? String

      if let albumArtworkData = information["kMRMediaRemoteNowPlayingInfoArtworkData"] as? Data {
          let image = UIImage(data: albumArtworkData)
          // Do something with artwork
      }

      print("\(songName ?? "Unknown Song") by \(artistName ?? "Unknown Artist") from \(albumName ?? "Unknown Album"). Playing in \(itemId ?? "Unknown Item ID") app")
  }
}

This code will ask the system for the currently playing media's info, including artist, album, song, and artwork. It uses the same info that the control center's now playing display uses, so any apps that use that (Apple Music, Podcasts, Spotify, etc.) will provide the right info.

Youtube, however, does not show up in the control center when playing, and thus it's now playing info cannot be accessed using this method. Perhaps there's a picture-in-picture equivalent that could be used, but I'll save that for another day. I have not tested the Youtube Music app.

The one missing link for this solution is getting notified when media changes (skipped, paused, ended, etc.) I saw in the library of private frameworks some notification symbols that could be used like kMRMediaRemoteNowPlayingApplicationDidChangeNotification, but I'll have to test those later.

Let me know if you have questions, I'm happy to help

MichaelJoo commented 1 week ago

Thank you so much Jordan for all the detailed insight you provided. I have raised a ticket to Apple support, although I have low expectation of them giving me the right information.

What you have wrote on being able to populate such info using private APIs, aligns with the current state of the apps, I didn't see any apps having this feature apart from being able to integrate with Apple Music and Spotify. The only thing I can probably do now is to integrate further with Spotify using their SDK, as my workout app is for commercial usage.

As a programmer, I can tell your codes are well structured and you have the rigor to dig into the necessary codes, which I admire.

Having said that, I will dig in further time to time, and update you if I get anything interesting from Apple support :) Will share Spotify SDK integration in future as well. Kindly close this ticket, if needed :) Great work on this repo btw, looks beautiful.