Roadmap

[kellnerd]

This is just a loosely ordered list of things I already have on my radar, to be cleaned up later™️.

Harmonized Data

• [x] Merge missing properties into the preferred provider's data
• [ ] Check for conflicting properties during merge
  • [ ] Warning for duration
  • [x] Error for GTIN
  • [x] Error for incompatible medium track counts
  • [ ] Merge tracks if the total track count matches (1 medium vs N media)
  • [x] Skip missing tracklist! (by far the most common error in the test phase logs)
  • [ ] Merge empty medium into medium with tracks
• [ ] Release date quality ranking, plausibility checks for each provider (new attribute "date.warning"), merge strategy "prefer latest"
• [ ] Generally warn about pre-release data
• [ ] Guess featured artists from titles
• [x] Copyright notices
• [ ] Explicitness of tracks
• [ ] Optional title cleanup (numeric prefix, ETI style etc.)
  • [ ] Deezer allows crediting the same artist multiple times
• [ ] Customizable search & replace rules
• [ ] Audiobook / audio drama mode?
• [x] Preserve catalog numbers
• [ ] Improve language detection (skip too short inputs, try alternatives?)

Providers

• [x] iTunes: Ensure that collection.trackCount equals the number of returned tracks
• [x] iTunes: Warn about responses which contain multiple release variants for an UPC
• [x] iTunes: Country list for optional region lookups
• [x] iTunes: Use region from URL
• [ ] iTunes: Try to use artist (or ISRC?) region for canonical (region-specific) URL
• [ ] iTunes: Try next region instead of throwing if JSON parsing fails (see screenshot)
• [x] Spotify: Pad GTIN with zeros if no results are found (example)
• [x] Deezer: Truncate padded GTIN
• [x] Bandcamp: Try band URL as label URL
• [x] Bandcamp: Add untitled hidden tracks, only their count is available as OG meta header
  • [x] Extract more than just trAlbum into snapshots -> wrapper object, avoid deserializing JSON
• [x] Bandcamp: Try band URL as label URL, extract label from packages
• [x] Bandcamp: Check whether band is part of the release artist before using it as label
• [x] iTunes: Warn about missing tracklist
• [x] Bandcamp: VA releases
• [x] Bandcamp: /track URLs (#7)
• [ ] Bandcamp: Extract release ISRCs from all /track URLs (expensive, only if there is no better source)
• [ ] iTunes: Drop " - Single" from title (#9)
• [x] Bandcamp: Extract track images, from embedded player (only for pre-releases so far)
• [ ] #46
• [ ] iTunes: Show not only the URL with the last region when all lookup attempts failed
• [ ] Bandcamp: no tracks https://2nxmusic.bandcamp.com/album/stolen-lullabies
• [x] Bandcamp: custom domains (#8)
• [ ] Deezer: API sometimes returns too many tracks: https://musicbrainz.org/edit/112474481 or https://www.deezer.com/album/303245

MusicBrainz

• [ ] Suggest existing release group
• [ ] Find release group or similar releases, reuse recordings? Similar query as MB duplicates tab?
• [x] Resolve external links to MBIDs
  • [x] Don't resolve ambiguous URLs to MBIDs
  • [x] Allow two URL rels if for the same target entity (e.g. download and streaming)
  • [ ] Cache pending requests in a map, parallel resolving of all identifiers
  • [x] Use resolved MBID of release artist for unresolved but identically named track artists
  • [ ] Combine release and track artists which share identifiers or names to avoid inconsistent results in edge cases
• [ ] Guess release group types
• [x] Create edit note
  • [x] Add permalink / homepage / repository URL (and version?)
• [ ] Optionally fill the annotation with additional data (make the sections configurable)
  • [x] Copyright notice
  • [ ] Availability
  • [x] Release ~and track level~ credits (text only so far)
  • [ ] Explicitness (show, but do not seed for tracks; add to release disambiguation?)
• [ ] Detect European releases (special country XE)
• [ ] Target seeder at existing release
• [x] Use ampersand for last joinphrase by default
• [ ] Support track URLs for other providers and suggest to look their release up

Infrastructure

• [x] URL lookup -> GTIN -> parallel GTIN lookups
• [x] Support provider-specific messages
• [x] Return all provider error messages if no lookup was successful
• [x] Allow to choose and exclude providers (Provider preferences)
• [ ] Allow providers to return multiple releases, i.e. different variants (e.g. for Bandcamp)
• [x] Cache management: https://github.com/kellnerd/snap_storage
  • Invalidation strategy: FIFO/LRU/TTL? Maximum age (optional)
  • In memory and/or long-time cache? JSON files ~with compression or Redis?~
  • [x] Cache multiple versions with timestamps (daily? only if there have been changes?)
  • [x] Let the requester know how old the data is and whether it is from the cache
  • [x] Permalinks to specific cached version (include GTIN, enabled providers, optional additional URLs or ProviderName=ProviderId pairs)
• [x] Optimize lookups (perform no GTIN lookup if ID was already looked up)
  • These repeated lookups also skew the calculated processing time for the initial provider (e.g. Deezer track requests are now cached)
• [ ] Use as few requests as possible (only make additional API calls for a provider if data is missing, e.g. iTunes regions or Deezer ISRCs)
• [ ] Lookup by metadata (label and catno, title, artist, track count etc.) for providers without GTIN
• [ ] Create provider feature categories (e.g. streaming, physical, with GTIN/ISRC, GTIN lookup, scraper, audio drama, Japanese etc.)
• [ ] Lookup the entire discography of a given artist/label
• [x] Make MusicBrainz base url configurable (environment variable)
• [x] Deduplicate lookup ReleaseOptions.regions option by using an ordered set
• [x] Manage lookup state: Each provider "Example" is split into two classes ExampleProvider and ExampleReleaseLookup, where ExampleReleaseLookup has a (readonly) property provider
  • Splits general request logic and release processing logic
  • Possible to store release lookup state as class properties
  • Separation of unrelated tasks once we add artist/label lookups later, e.g. as ExampleArtistLookup and ExampleLabelLookup
• [ ] Warn that available regions may not be accurate before the release date has passed (anywhere on earth, UTC-12)
• [ ] Extract provider URLs from link shortener pages
• [ ] Extract provider IDs and GTIN from a-tisket URLs
• [ ] Write more test cases...
• [x] Preserve URL blurb (for Beatport)
• [x] Improve logging of AggregateErrors, they make it a PITA to find the real issue

Web Interface

• [x] Display header with logo and description
  • [x] Harmony: Music Metadata Aggregator and MusicBrainz Importer/Seeder
  • [x] Design banner logo and icon
• [x] Display footer with version, repo URL and support URL (environment variables DENO_DEPLOYMENT_ID, REPO_BASE_URL, optional COMMIT_BASE_URL, SUPPORT_URL)
• [x] Add OpenGraph meta tags
• [x] Allow to choose and exclude providers (persistent provider checkboxes)
• [ ] Persist preferred regions input
• [x] Show provider and alternative values for interesting properties
  • [ ] Improve track length comparison, Deezer truncates instead of rounding
• [ ] Settings page/section with persisted checkboxes
• [ ] Multiple URL inputs (dynamic form)
• [ ] Provider URL detection on the frontend (URLPattern polyfill for Firefox and Safari? https://caniuse.com/mdn-api_urlpattern)
• [x] CSS
  • [ ] Responsive grid
  • [x] Beautiful icons: https://tabler.io/icons enhanced by a few custom ones in the same style
• [x] Provider icons (~external links or data URIs? inline TSX SVG?~ SVG sprite built with TSX)
• [x] Post-submission route/page ("release actions"):
  • [x] ISRC submission (kepstin/tatsumo/custom?)
  • [ ] Artwork (ECAU)
  • [ ] External links (for artists, maybe for labels?)
• [x] Dynamic region list display: count, compact flags, detailed list
• [ ] Group regions by continent
• [ ] Serve documentation, written in Markdown
• [x] ~Use HTTPS by passing key and cert options to start()~ Support X-Forwarded-Proto proxy header
• [x] Trim GTIN input to avoid unnecessary errors