Rewrite External Metadata

[Arthi-chaud]

Fetching external metadata takes a long time, and multiple runs are sometimes necessary to get all metadata. Additionally, we usually fetch more metadata than necessary.

Let's rewrite how this whole process works:

API (See #760)

External Metadata

There should be a single model for external metadata with:

• [x] Description Text
• [x] Rating (for albums)
• [x] Sources: An array of URLs that can be accessed by the final user (e.g. the actual Genius page, not the API endpoint)
  • [x] Each source should be associated with the relevant provider
• [x] An endpoint should be exposed to allow the MS to POST this data
  • [x] The endpoint should be protected, and only accessible to the MS

Providers

We should keep the provider model, but let's make it simpler

• [x] A Name
• [x] An Icon
• [x] An endpoint should be exposed to allow the MS to POST the resource
  • [x] The endpoint should be protected, and only accessible to the MS
• [x] An endpoint should be exposed to allow the MS to POST the icon

(I.e. Remove the homepage URL)

Matcher Microservice

A microservice should be dedicated to this feature:

Event handling

On Artist/Album/Song creation:

• [x] The API should publish an event on a message queue (#764)
• [ ] The MS should receive that event, fetch necessary data from API (such as the artist for the album/song)
• [ ] POST the collected metadata to the API
• [ ] POST the lyrics to the API
• [ ] POST the song/album's genres to the API

HTTP API

• [ ] The MS should have an HTTP API to request specific artist/album/song refresh

Providers

• [ ] The list of providers should be hardcoded
• [ ] If a provider is not present in the API, the MS should create it
  • [ ] And POST its icon

Front

• [ ] The front should use the new data model

Conception

• [x] What language should this MS be written with?
  • Answer: Python. (I would have preferred Haskell, but Python will definitely be more maintainable and it's an opportunity to do a mid-sized project with that language)
• [ ] The MS will use the AcoustID of the source file to find its entry in Musicbrainz (See #762)

[Arthi-chaud]

Here is the workflow of the metadata matcher:

• [ ] On 'Artist Created'
  • If MusicBrainz is enabled, look up
  • If multiple artists match, rerun the query using one of their songs
  • If artist match
    • Get MBID of artist
    • Get Description of artist
    • Match each 'External Links' with each enabled providers
  • For each enabled provider (that has not been linked yet), look artist up
  • If no description was found, get it from one of the linked providers
  • Get an image of the artist using one of the linked providers
• [ ] On 'Song Created' Event
  • If MusicBrainz is enabled, look up
  • If song match
    • Get MBID of song
    • Get Description of song
    • Get Genres of song
    • Match each 'External Links' with each enabled providers
  • For each enabled provider (that has not been linked yet), look artist up
  • If no description was found, get it from one of the linked providers
  • If no genre was found, get it from one of the linked providers
  • Get lyrics using one of the linked providers
• [ ] On 'Album Created' Event
  • If MusicBrainz is enabled, look up
  • If album match
    • Get MBID of album
    • Get Description of album
    • Get Genres of album
    • Match each 'External Links' with each enabled providers
  • For each enabled provider (that has not been linked yet), look artist up
  • If no description was found, get it from one of the linked providers
  • If no genre was found, get it from one of the linked providers
  • Get rating of the album using one of the linked providers

The goal is to have the smallest number of HTTP request per item, for the sake of performance and network load

[Arthi-chaud]

Regarding the collection of providers, at startup

• [ ] Load settings, filter out disabled providers
• For each enabled provider
  • [ ] If it exists in the API: resolve its ID
  • [ ] Else: POST it, along with an icon, and get its ID