[More data] Enrich database through information from MangaUpdates

[Koopzington]

As manga requirements often involve bits of information that's not available on AniList but largely on MangaUpdates, one should explore the possibilities of retrieving data through their API. Since AniList doesn't store MangaUpdates IDs one'd first have to establish a link between AL IDs and MU IDs.

• MangaDex does have both IDs but unfortunately their API doesn't allow querying entries by external IDs.
• WikiData let's one query a list of all manga containing external IDs from both sites but as of today (2023-01-02) there are only 2690 results, which is pretty small considering AniList has over 87k manga entries and MangaUpdates supposedly over 200k entries

  SELECT DISTINCT ?alMangaId ?muMangaId WHERE {
  ?item wdt:P8731 ?alMangaId.
  ?item wdt:P11149 ?muMangaId.
  }

• It might yield more results to scrape MangaUpdates directly and do string comparisons with the native titles of AniList

Information from MangaUpdates that a lot of users would probably look forward to:

• Publisher
• Magazines it was published in
• Licensing status and english publisher
• Is it completely scanlated/available in english (personally mostly interested in this 'cause i have a gift of planning to read manga which's scanlation has been abandoned several years ago)

[Koopzington]

Since then the backend has been extended by:

• A scraper for MangaUpdates that retrieves all base information that might be useful
• A scraper for mappings from the Animeshon API which on import is creating mappings over their MangaDex and MyAnimeList IDs (31,399 mappings)
• A scraper for mappings from MangaDex (27,505 mappings)
• A mapper which let's users manually map AL entries to MangaUpdates entries (1,028 mappings so far)

Current total coverage: 59,932/94,138 AL manga Total manga on MangaUpdates: 186,406

The aforementioned own mapper is currently randomly picking unmapped AL manga and searches the native title on a Meilisearch instance fed with the data from the MU scraper. After using it myself for a while i noticed/confirmed the following things:

• You can't rely on 100% title matches
• MU titles can contain "(Novel)" to differentiate between a manga and LN of the same franchise despite having types
• MU titles can contain the author's name to differentiate between works with the same title
• AL manga (especially One Shots) can be part of an MU manga (e.g. Anthologies or all works of an author in a single Tanko)
• AL manga can be mapped against multiple entries on MU (example: Seiza no Onna AL, MU#1, MU#2, MU#3)
• Staff names can be different between AL and MU (e.g. "o" instead of "ou" or a different pen name entirely)
• Staff names on MU Manhua can be written in Kanji

With the made observations i wouldn't take the chance of automating any of this and instead let users do it manually. As for the implementation of that I will require users to Login into AniList through OAuth to prevent any malicious intentions. Suggested Mappings will also not directly get imported but instead have to be confirmed by other users (a specific amount has yet to be determined) first.

The chances of a title matching is pretty high when the covers are identical. It might be worthwhile to read a bit about comparing images and incorporating the visual similarity of the covers into the suggestions.

The WikiData query delivers 2,812 results now. I have yet to check whether adding them would be worth it.