False positive duplicates getting removed from API

tie commented 2 months ago

Description

Certain TV series contain identical TheMovieDB IDs and get deduplicated on update. That is, the API returns null for some series (including popular ones like the ongoing Mushoku Tensei S2P2 and Konosuba S3).

Some of the themoviedb IDs appear to be introduced in https://github.com/Fribb/anime-lists/commit/0d22a783.

Example

Title: Kono Subarashii Sekai ni Shukufuku o! AniDB: https://anidb.net/anime/11261 API: https://arm.haglund.dev/api/ids?source=anidb&id=11261

{"anidb":11261,"anilist":21202,"myanimelist":30831,"kitsu":10941}

but subsequent seasons are not present in the API responses,. e.g.

Title: Kono Subarashii Sekai ni Shukufuku o! 2 AniDB: https://anidb.net/anime/11992 API: https://arm.haglund.dev/api/ids?source=anidb&id=11992

null

tie commented 2 months ago

This could be caused by the recent refactoring in https://github.com/Fribb/anime-lists-generator, cc @Fribb. I don’t think these titles are even supposed to contain TheMovieDB IDs.

Fribb commented 2 months ago

In the first iteration of the generator, TheMovieDB only covered the Movie Side of things so it made sense to only query or look up and add the TheMovieDB ID to entries that are a Movie.

However, for a while now, TheMovieDB also provides TV Shows on their end and someone made a request that a Show should also contain a TheMovieDB ID if available. This was the reason for the generator being refactored.

But the list itself is still based on individual releases like MyAnimeList and a couple of other Databases do it. This means that the individual Seasons and releases of, for example, KonoSuba can have the same TheMovieDB and TheTVDB IDs because those sources don't list them as individual releases but as "combined" one (One show, multiple seasons).

here is an example for KonoSuba Season 1 + 2 and an OVA.

 {
    "livechart_id": 1615,
    "thetvdb_id": 303867,
    "anime-planet_id": "konosuba-gods-blessing-on-this-wonderful-world",
    "imdb_id": "tt5370118",
    "anisearch_id": 10448,
    "themoviedb_id": 65844,
    "anidb_id": 11261,
    "kitsu_id": 10941,
    "mal_id": 30831,
    "type": "TV",
    "notify.moe_id": "NZ8lpFiig",
    "anilist_id": 21202
  },
  {
    "livechart_id": 2077,
    "thetvdb_id": 303867,
    "anime-planet_id": "konosuba-gods-blessing-on-this-wonderful-world-2",
    "imdb_id": "tt5370118",
    "anisearch_id": 11210,
    "themoviedb_id": 65844,
    "anidb_id": 11992,
    "kitsu_id": 11937,
    "mal_id": 32937,
    "type": "TV",
    "notify.moe_id": "vVO6tFmig",
    "anilist_id": 21699
  },
  {
    "livechart_id": 3359,
    "thetvdb_id": 303867,
    "anime-planet_id": "konosuba-gods-blessing-on-this-wonderful-world-2-ova",
    "imdb_id": "tt5370118",
    "anisearch_id": 12616,
    "themoviedb_id": 65844,
    "anidb_id": 13317,
    "kitsu_id": 13136,
    "mal_id": 34626,
    "type": "OVA",
    "notify.moe_id": "WmpG2Kmmg",
    "anilist_id": 97996
  },

They all have different mal_ids but the same themoviedb_id and thetvdb_id because on TMDB and TVDB they are one show, on MAL they are different releases. So yes, they should have a TheMovieDB to get the relation between the MAL release for KonoSuba Season 2 and the Show on TheMovieDB.

beeequeue commented 2 months ago

Then I'll just do what I did with thetvdb and add a separate endpoint for querying their IDs since the normal ones assume one entry per ID

beeequeue / arm-server

False positive duplicates getting removed from API #789

Description

Example