sven-7
commented
3 years ago Thinking on this, but bringing @EndOfLine369 and @reconman into this.
Open ZeroQI opened 3 years ago
sven-7
commented
3 years ago Thinking on this, but bringing @EndOfLine369 and @reconman into this.
sven-7
commented
3 years ago First, the lift of moving to TheMovieDB is probably extremely heavy on the Anime-Lists side of this. I could be wrong, but that is my first take. Second, it seems like they have decent metadata so if we do find a way to either A) incorporate TheMovieDB for TV shows in general or B) switch over -- I think we'll be in decent shape.
Before committing to that, figuring out the mapping is critical. I do see in their metadata that they have TheTVDB ID as a field that can be filled out. We might be able to work with that?
My biggest issue with other options Sonaar, Shoko, etc. is the Absolute Numbering and automatic season mapping that HAMA does so well. There are a few thoughts I have on this part to make sure we don't lose it.
Edit:
Question - Do we ALL need a TVDB license or can this operate off of one?
reconman
commented
3 years ago Keeping anime-lists compatible with TVDB AND TMDB is impossible in my opinion. If we want to accomplish that, we would need a mapping structure similar to thexem, which means
Maybe the folks at thexem are also thinking about the TVDB problem. I'm thinking that maybe thexem could include additional databases like IMDB and TMDB in their mappings, but the codebase is probably hard to maintain, so I don't expect them to be able to adjust easily. Additionally, it would restrict the number of users who can edit the mappings, since user registrations are closed.
@thezoggy Since you're the only one maintaining the xem code base, what's your input on enhancing thexem with additional databases and on the TVDB deprecation?
ZeroQI
commented
3 years ago TheTVDB series121,161, movie153,237 person510,784 [https://thetvdb.com/search?] TMDb tv shows 107,282, movies 633,023. tv episodes 2,452,230. [https://www.themoviedb.org/faq/general] TVrage shows 3180 [https://www.tvrage.com/all/]
Comparison page TMDB/TVDB: https://trakt.tv/transition/tmdb/missing
Choices are
Get tvdbid
https://api.themoviedb.org/3/tv/1399/season/1?api_key=7f4a0bd0bd3315bb832e17feda70b5cd&append_to_response=external_ids
"external_ids":{"freebase_mid":"/m/0gmd1gd","freebase_id":"/m/0gmd1gd","tvdb_id":364731,"tvrage_id":null}}
freebase ended in 2015 [last dump https://developers.google.com/freebase] so useless
'Find' by tvdb id:
https://api.themoviedb.org/3/find/85083?api_key=7f4a0bd0bd3315bb832e17feda70b5cd&external_source=tvdb_id
{"movie_results":[],"person_results":[],"tv_results":[{"genre_ids":[99],"original_language":"en","poster_path":"/cbzbvNe8ht9jSHKsNPXfx5I9BWi.jpg","first_air_date":"2009-02-11","vote_average":8.5,"original_name":"Nature's Great Events","origin_country":["GB"],"id":19744,"vote_count":21,"overview":"Documentary series looking at the most dramatic wildlife spectacles on our planet, showing how life responds to natural events which can dramatically transform entire landscapes.","backdrop_path":"/e0onhW0lUH4BKMiyq4vNUSL5gVo.jpg","name":"Nature's Great Events","popularity":6.776}],"tv_episode_results":[{"air_date":"2004-02-22","episode_number":6,"id":272662,"name":"Heavy Support Vehicles","overview":"","production_code":"","season_number":4,"show_id":3890,"still_path":"/q1tcCgXiGAhsk2tvg05fwQzvhK.jpg","vote_average":0.0,"vote_count":0}],"tv_season_results":[{"air_date":"2009-02-11","episode_count":6,"id":30012,"name":"Season 1","overview":"","poster_path":"/cbzbvNe8ht9jSHKsNPXfx5I9BWi.jpg","season_number":1,"show_id":19744}]}Extras: https://github.com/vmirage/TMDBTrailer.bundle/blob/master/Contents/Code/__init__.py could add extras
sven-7
commented
3 years ago Get tvdbid https://api.themoviedb.org/3/tv/1399/season/1?api_key=7f4a0bd0bd3315bb832e17feda70b5cd&append_to_response=external_ids "external_ids":{"freebase_mid":"/m/0gmd1gd","freebase_id":"/m/0gmd1gd","tvdb_id":364731,"tvrage_id":null}} freebase ended in 2015 [last dump https://developers.google.com/freebase] so useless
Perhaps we can use this TheTVDB ID to bridge the gap between the three? On the one hand, $12/year doesn't truly bother me (aside from I disagree with what they're doing), especially if they make contributors free, then likely $0 option can be palatable.
ZeroQI
commented
3 years ago Issue is order of eps can be different and dual episodes can be split or not. Other issue is, let's say we do go to TMDB, will all added bandwidth, it might have to raise funds and go the same route...
We could just use my code and do a separate TMDB only agent, managing movies and series. That would allow to test TMDB, be useful for documentaries as like anime they mix movies and series in a single library and would minimise the time to port it to hama, and allow people to add what is not on TheMovieDb...
Tyestor
commented
3 years ago I would rather hama somehow completely ditch thetvdb/themoviedb/imdb etc in my opinion. They are simply not made for anime. An anime agent should strive to use databases/apis specifically designed for anime, that means: MAL, AniDB, AniList etc etc. Specifically, the way TheTVDB defines anime in its databases is simply not the way the anime community does and it causes WAY too many problems.
Anyway, that's my two cents.
ZeroQI
commented
3 years ago TheTVDB have episode summaries in multiple language when AniDB had no summary then now english only episode summaries... We have a mapping file to match both which only needs updating when an issue occurs...
Not everybody is fan of AniDb numbering and there is more that one way in the 'Anime community'
Furthermore we need an infrastructure that is the best we can do given the circumstances, as metadata can be added...
Why are you not using the following agents (especially AniDb):
If you tell me the metadata is lacking on a single database, I would recommend to add the missing meta, anyway, that's my two cents.
ZeroQI
commented
3 years ago https://support.thetvdb.com/kb/faq.php?id=82
Your user subscription will also provide you with a legacy v3 API key that you can use to access the v3 API until the project you’re using switches their code to support user PINs, or until the v3 API is deprecated in mid-2021.
http://thexem.de/ is anime specific and could be used if TMDB matches the scene...
EndOfLine369
commented
3 years ago So......there are multiple items to discuss:
TVDB V4 migration: This should be done. There will be people that are willing to pay for the key or try to get a free key. It would be changed like OMDB where the key is a pref field. I personally will be trying to get a free key and might even fork out for it only if they provide a more secure way then providing over my credit card info (paypal, google pay, apple pay, etc...)
TVDB API Key: Has anyone tried to contact Plex about using their key? Or maybe a service they will provide through Plex? They currently provide a web proxy to the tvdb2 api. Will there be a service of theirs that we can call to pull TVDB info via Plex? Maybe even a Plex Pass functionality they will provide. Or talked to TVDB about keys for OpenSource projects?
Dependence on TVDB: Too many modes REQUIRE TVDB info for new season/ep transformation. 'anidb2'(inserted as tvdb) - Requires season/ep order info 'anidb3'(inserted as tvdb) - Requires season/ep order info 'anidb4'(inserted as tvdb6) - Requires season/ep order info 'tvdb2' - Requires season/ep/absolute ep order info 'tvdb3' - Requires season/absolute ep order info 'tvdb4' - Requires absolute ep order info 'tvdb5' - Requires ep air date info Ignoring 'tvdb' that doesn't require info for transformation. Just goes with whatever number is on the file. That leaves only 'anidb', 'imdb', 'tmdb', & 'tsdb'(that's never used) that don't at some point require TVDB info
What we would require if TVDB is not available for multi season setups: season/ep order info (Provided by TMDB) absolute ep order info (Does TMDB provide this info? If not, the only one we could do is our own abs number calculation) ep air date info (Provided by TMDB)
Whatever system we use to get item 4 above, we would need mapping of AniDB to that system's id entries:
Can 'anime-lists' be updated to handle a separate source for mapping?
It already stores IMDB (imdbid) & TMDB (tmdbid).
There could be a mention of overhead of adding in this additional mapping but honestly, most of the entries don't require "mapping-list" entries. And the ones that do should just be relied on by users to provide and only actively handle new entries from the start of this setup on.
One problem that has already been raised is that TMDB reuses numbers. A movie & series can have the exact same number. The TMDB in 'anime-lists' can only be a movie number. We can simply add a TMDB field (tsdbid) for series id numbers along with default season (tsdbdefaultseason) & ep offset (tsdbepisodeoffset) attributes.
EX <anime anidbid="" tvdbid="" defaulttvdbseason="" episodeoffset="" tmdbid="" tsdbid="" tsdbdefaultseason="" tsdbepisodeoffset="" imdbid="">
Note, even if we could find a system that would give us the TMDB series id from a TVDB id, it would still be missing the TMDB specific season & episode mapping that could easily not be the same as TVDB.
The only thing missing at that point would be the possible 'mapping-list' details.
Currently, these are the fields that are possible:
<name>Gintama'</name>
<mapping-list>
<mapping anidbseason="0" tvdbseason="0">;1-6;</mapping>
<mapping anidbseason="0" tvdbseason="0" start="1" end="6" offset="16"/>
</mapping-list>
<supplemental-info>
<studio>Sunrise</studio>
</supplemental-info>An additional "mapping-list-tsdb"/"mapping-tsdb" tags & "tsdbseason" attribute with info could be added. Apps should be looking for specific xml child tags by name so by adding a new named child I would think not interfere with any application.
<mapping-list-tsdb>
<mapping-tsdb anidbseason="0" tsdbseason="0">;1-6;</mapping>
<mapping-tsdb anidbseason="0" tsdbseason="0" start="1" end="6" offset="16"/>
</mapping-list-tsdb>Or even just adding the new "mapping-tsdb" child tags into the existing "mapping-list" group w/diff "tsdbseason" attribute. This would be a little riskier as could potentially cause issues in apps plainly iterating over the "mapping-list" childs.
Adding a different "mapping-tsdb" tag & attribute:
<mapping-list>
<mapping anidbseason="0" tvdbseason="0">;1-6;</mapping>
<mapping-tsdb anidbseason="0" tsdbseason="0">;1-6;</mapping>
</mapping-list>
OR having the same tag but different attribute
<mapping-list>
<mapping anidbseason="0" tvdbseason="0">;1-6;</mapping>
<mapping anidbseason="0" tsdbseason="0">;1-6;</mapping>
</mapping-list>This would also imply separate modes in ASS/HAMA that would do the season/ep transformation to TSDB in the different display ways we currently have. 'anidb5', 'anidb6', 'anidb7' (mirrors of anidb2/3/4) 'tsdb2', 'tsdb3', 'tsdb4', 'tsdb5' (mirrors of tvdb2/3/4/5)
We could also have an anidb mode that behaves like tvdb4. Just breaks up the anidb series into our own predefined seasons based off AniDB episode numbering (instead of tvdb abs ep numbering).
ZeroQI
commented
3 years ago Since TheTVDV is mapped to anidb it is used in all modes (anidb-tvdb) but required for anidb2-5 and then provide summaries in other languages which and can't... Therefore for summaries in other languages and mapping it is pretty needed, especially for users that used the mapping modes
There is a episode numbering field for TheTVDB or TMDb in new plex series agent, so there might be a way to map... How does the tvdb does it? Or it just use TMDb metadata when using episode numbering and no eta? If somebody had the agent code link/path, i am interested to check
TMDb have very precise rules for anime which should limit bickering and TVDB locking on an arbitrary order https://www.themoviedb.org/bible/tv/59f743289251416e71000037
TMDb api does support 7 types for ordering https://developers.themoviedb.org/3/tv/get-tv-episode-groups https://www.themoviedb.org/bible/tv/59f743289251416e71000037]
These mapping are interesting, and letting all these numberings to be chosen by the user worthwhile, but might warrant a new agent doing movies and series using TheTVDB (for documentaries, anime) or a subgroup [tsdb1-xxx] to [tsdb7-xxx] to allow for weird series to switch to TMDB series useful numbering...
dunno about TVDB API v2 proxy or how to use API key, i do have one though
sven-7
commented
3 years ago TVDB V4 migration: This should be done. There will be people that are willing to pay for the key or try to get a free key. It would be changed like OMDB where the key is a pref field. I personally will be trying to get a free key and might even fork out for it only if they provide a more secure way then providing over my credit card info (paypal, google pay, apple pay, etc...)
I'm 100% with you here. I think we need to support this. I'll also try to contribute and get a free key.
Whatever system we use to get item 4 above, we would need mapping of AniDB to that system's id entries: Can 'anime-lists' be updated to handle a separate source for mapping? It already stores IMDB (imdbid) & TMDB (tmdbid). There could be a mention of overhead of adding in this additional mapping but honestly, most of the entries don't require "mapping-list" entries. And the ones that do should just be relied on by users to provide and only actively handle new entries from the start of this setup on. One problem that has already been raised is that TMDB reuses numbers. A movie & series can have the exact same number. The TMDB in 'anime-lists' can only be a movie number. We can simply add a TMDB field (tsdbid) for series id numbers along with default season (tsdbdefaultseason) & ep offset (tsdbepisodeoffset) attributes. EX
Note, even if we could find a system that would give us the TMDB series id from a TVDB id, it would still be missing the TMDB specific season & episode mapping that could easily not be the same as TVDB. The only thing missing at that point would be the possible 'mapping-list' details. Currently, these are the fields that are possible:
@EndOfLine369 I was talking with someone yesterday bout the idea of quering TheMovieDB's API via the External ID field for TheTVDB.
https://developers.themoviedb.org/3/getting-started/external-ids
In my mind, we could use that to snapshot Anime-Lists with a 'tsdb' field. It might not be perfect, but will do a lot of the lifting. Alternatively/additionally, ASS/HAMA might be able to do the same, if Anime-Lists continues to stay updated.
EndOfLine369
commented
3 years ago @ZeroQI, Good to know the different ordering options from TMDB are available.
TVDB also provides different ordering output via "/series/{id}/episodes/{season-type}" now in their latest api. That's what you now see on the website post their last major upgrade. We just do it ourselves by sorting and pulling out the details to override ep number.
For TMDB, I think we should keep it at only Official, Absolute, Arc, & Air Date like we currently do for TVDB. Arc would depend if their Arc groupings are reasonable or not and we have to just create our own mapping files like we did for TVDB4 for seasons & images. Just create the mirrors (as mention is item 6 above) of what we already provide from TVDB. New orders would make this work more complicated than this transition work should require. Other modes can be created later, if even wanted, after the addition of TMDB for other episode order options.
Trying to separate out to a different agent doesn't seen realistic. It would require code to be duplicated from HAMA and then kept in sync. As well as would remove the ability to have multiple sources as options for fields. If kept all in one, it would even allow for TVDB & TMDB to be both possible source options for a series. Like episode titles from TMDB but episode summaries from TVDB. That's the whole major advantage of HAMA. Multiple sources in one agent 😁.
If we can agree on the above, its just a matter then of the where/how we get the AniDB -> TMDB mapping. Getting a TMDB id from a TVDB id realistically only works when they are single season series and don't have issues in S0 orders not matching like does happen in AniDB<->TVDB. And any more than just a S1 will almost guarantee a many-to-one mapping by seasons or ep offset. As well as a hard requirement for any of our remapping modes utilized when they are many-to-one.
EndOfLine369
commented
3 years ago Whatever system we use to get item 4 above, we would need mapping of AniDB to that system's id entries: Can 'anime-lists' be updated to handle a separate source for mapping? It already stores IMDB (imdbid) & TMDB (tmdbid). There could be a mention of overhead of adding in this additional mapping but honestly, most of the entries don't require "mapping-list" entries. And the ones that do should just be relied on by users to provide and only actively handle new entries from the start of this setup on. One problem that has already been raised is that TMDB reuses numbers. A movie & series can have the exact same number. The TMDB in 'anime-lists' can only be a movie number. We can simply add a TMDB field (tsdbid) for series id numbers along with default season (tsdbdefaultseason) & ep offset (tsdbepisodeoffset) attributes. EX Note, even if we could find a system that would give us the TMDB series id from a TVDB id, it would still be missing the TMDB specific season & episode mapping that could easily not be the same as TVDB. The only thing missing at that point would be the possible 'mapping-list' details. Currently, these are the fields that are possible:
@EndOfLine369 I was talking with someone yesterday bout the idea of quering TheMovieDB's API via the External ID field for TheTVDB.
https://developers.themoviedb.org/3/getting-started/external-ids
In my mind, we could use that to snapshot Anime-Lists with a 'tsdb' field. It might not be perfect, but will do a lot of the lifting. Alternatively/additionally, ASS/HAMA might be able to do the same, if Anime-Lists continues to stay updated.
We could potentially setup a dump file like we do for missing episodes/summaries, and have it contain info on when there is a TVDB id but no TSDB id. If not this, what could you potentially be looking for from HAMA?
Getting one off bulk update of TMDB ids from TVDB id for tsdb ids would get probably 50% firmly sorted as most of the entries as have no special TVDB mapping other than just putting in a TVDB id at season 1 with no episode offset. Its the ones that have an ep offset, default season != 1, or mapping-list entries that would just be missing those details for TMDB series special mappings. You might want to potentially leave those unmapped with no tsdbid or otherwise add them knowing that they are missing season/ep offset/mapping details.
Looking at the currently cached "anime-list-master.xml":
So probably ~3000 entries that are not a basic 1x1 as some entries will have more than one of those flags.
Thoughts on the possible options of formats of enhancing 'anime-lists' to have it have both TVDB & TMDB mapping data? Its all the same format as current, just additional tags/attributes.
sven-7
commented
3 years ago Thoughts on the possible options of formats of enhancing 'anime-lists' to have it have both TVDB & TMDB mapping data? Its all the same format as current, just additional tags/attributes.
I'm personally all for it, @EndOfLine369. Regardless of TheTVDB changes, this will be an enhancement/improvement for metadata.
TVDB also provides different ordering output via "/series/{id}/episodes/{season-type}" now in their latest api. That's what you now see on the website post their last major upgrade. We just do it ourselves by sorting and pulling out the details to override ep number.
Side note -- does this mean the alternative ordering like Re:Zero's director's cut can be used?
ZeroQI
commented
3 years ago TVDB migration v4 agreed with API key in settings, near future but no ETA
The update engine translate the supported numbering (thetvdb, anidb) for each source, so it matches the files in Plex:
I do not see how to support more ordering modes easily. We could add tmdb support but:
https://github.com/ZeroQI/Hama.bundle/blob/master/Contents/Code/AnimeLists.py
The only mapping i could find are not usable yet
While adding 'tsdb' and anidb ids in the xml is easy to https://github.com/Anime-Lists/anime-lists, as you see from the mappings, either we redo anidb to tmdb series (let's call it tsdb) mapping, doubling the size of the file, OR we create another project (json?) to map TSDB to TVDB, which could be used by the community like anime-lists is used by Hama, filebot, etc...
Not only the data type (multi-season series) would be identical between tvdb and tmdb series most of the time, but other projects could benefit (read: update it as they would also use it). only the differences would be mapped
tvdbid='' tmdbid='' season[tvdb_season]={'tvdb_episode': (tsdb_season, tsdb_episode}, ...} or season[tsdb_season]={tsdb_episode: (tvdb_season, tvdb_episode}, ...}
EndOfLine369
commented
3 years ago The update engine translate the supported numbering (thetvdb, anidb) for each source, so it matches the files in Plex:
- for anidb numbering tvdb source will translate tvdb->anidb
- for tvdb numbering anidb source will translate anidb->tvdb
I do not see how to support more ordering modes easily. We could add tmdb support but:
- for anidb numbering tmdb source will translate tmdb->tvdb->aniDB or tmdb->anidb
- for tvdb numbering tmdb source will translate tmdb->tvdb
Its "anidb<->tvdb" (current mapping) & "anidb<->tsdb" (new mapping) Its just the simple matter of having two "mappingList" variables. One for "anidb<->tvdb" (mappingListTVDB) & another for "anidb<->tsdb" (mappingListTSDB). Since the format will stay the same in anime-lists, the format would stay exactly the same. Otherwise you'd have to try to combine them info one "mappingList" which would require a lot more work.
https://github.com/ZeroQI/Hama.bundle/blob/master/Contents/Code/AnimeLists.py
- Line 238: def tvdb_ep(mappingList, season, episode, anidbid='')
- Line 301: def anidb_ep(mappingList, season, episode)
Since the two "mappingList" variables are the same format, no new "def tvdb_ep" or "def anidb_ep" are required.
Just create an alias function that just calls "def tsdb_ep":
def tsdb_ep(mappingList, season, episode, anidbid=''): tvdb_ep(mappingList, season, episode, anidbid)
So if we have an anidb series: tvdb_ep(mappingListTVDB, season, episode, anidbid) - Gives TVDB mapped info tsdb_ep(mappingListTSDB, season, episode, anidbid) - Gives TSDB mapped info
And if we have a TVDB/TSDB, you can just use both functions in tandem. TVDB ep -> AniDB ep -> TSDB ep: -- TVDB -> anidb_ep(mappingListTVDB, season, episode) -> tsdb_ep(mappingListTSDB, season, episode, anidbid) TSDB ep -> AniDB ep -> TVDB ep: -- TVDB -> anidb_ep(mappingListTSDB, season, episode) -> tsdb_ep(mappingListTVDB, season, episode, anidbid)
Another thought instead of this is to just enhance the single mappingList to make it cleaner. This would obv take more rewriting though but doable.
FROM:
mappingList: {
'defaulttvdbseason': '1',
'defaulttvdbseason_a': False,
'possible_anidb3': False,
's1e1_mapped': False,
'season_map': {'12921': {'max': '1', 'min': '1'}, '14098': {'max': '2', 'min': '2'}, '14956': {'max': '3', 'min': '3'}, '15600': {'max': '4', 'min': '4'}},
'TVDB': {'s1': {'12921': '0'}, 's2': {'14098': '0'}, 's3': {'14956': '0'}, 's4': {'15600': '0'}},
'tvdbcount': 4}
TO:
mappingList: {
'name': 'Slime Taoshite 300-nen, Shiranai Uchi ni Level Max ni Nattemashita',
'studio': 'some studio'
'TVDB': {
'defaultseason': '1',
'defaultseason_a': False,
'episodeoffset': '0',
'possible_anidb3': False,
's1e1_mapped': False,
'season_map': {'15201': {'max': '1', 'min': '1'}},
'mapping': {'s1': {'15201': '0'}},
'count': 1},
'TSDB': {
'defaultseason': '1',
'defaultseason_a': False,
'episodeoffset': '0',
'possible_anidb6': False,
's1e1_mapped': False,
'season_map': {'15201': {'max': '1', 'min': '1'}},
'mapping': {'s1': {'15201': '0'}},
'count': 1}
}While adding 'tsdb' and anidb ids in the xml is easy to https://github.com/Anime-Lists/anime-lists, as you see from the mappings, either we redo anidb to tmdb series (let's call it tsdb) mapping, doubling the size of the file, OR we create another project (json?) to map TSDB to TVDB, which could be used by the community like anime-lists is used by Hama, filebot, etc...
Not only the data type (multi-season series) would be identical between tvdb and tmdb series most of the time, but other projects could benefit (read: update it as they would also use it). only the differences would be mapped
tvdbid='' tmdbid='' season[tvdb_season]={'tvdb_episode': (tsdb_season, tsdb_episode}, ...} or season[tsdb_season]={tsdb_episode: (tvdb_season, tvdb_episode}, ...}
We should never assume they match in any way unless stated in the mapping. If the mapping will even be setup to say they match. Better to have no meta then bad meta as its easier to spot and will prompt people to get the mapping data fixed.
ZeroQI
commented
3 years ago Better no meta than wrong one, true still saying the difference anidb-tvdb is greater than tvdb tmdb as they are both seasons based, but the modification suggested are simple
Seem like api key for developer don't come with required pin code https://app.swaggerhub.com/apis-docs/thetvdb/tvdb-api_v_4/4.3.2#/login/post_login
So a developer need to pay for subscription to have a PIN code to do the development using TheTVDB v4 API bringing them revenue? Hell no...
alejakun
commented
3 years ago Am I the only one wondering about what will happen to the integrations/setups beyond just plex/hama? I'm freaking out that my setup that I just just got to work kind of the way I want will break if different approaches are taken from different projects, specifically I'm thinking in Sonarr/Radarr. Is this a general concern or am I way out of line for pointing this out?
thezoggy
commented
3 years ago most of the big apps worked with tvdb directly to work stuff out on how to deal with v4. with a lot of them doing a loadbalancer/proxy type approach where they can control the apikey on their side (otherwise keys would get leeched and used for others). the other apps are probably going to have to do the add field and push the issue off to the end user to get a key/contribute to get access.
now kodi for example years ago switched default tv scraper away from tvdb, so users also could just not use tvdb on that side.. and as more people switch to moviedb/tvmaze/whatever you will see better tv data there or requests to go that direction.
now, tvdb v4 did not launch as plan (no one expected it to)... so everyone is doing the wait and see approach. as v4 requires apps to redo their api/integration and v4 api isnt feature parity with v3.. so no one really wants to switch right now.
--
now where does xem stand on all this, the need for xem was supposed to go away with v3 api as tvdb was supposed to add the whole flexible season listing.. but it never came. maybe with v4 xem wont really be needed there. also as more of 'xem' type stuff is added to sonarr directly, the need for xem there also diminishes.
btw 'master' in xem is arbitrary as your just using it as a reference point between two entities. also master is only needed to do manual episode mapping. the entities on each end could be whatever that is in the database (tvdb/scene/anidb - tvrage/trakt are disabled). years ago there was call to add tvmaze to xem but its data was so poort i did not want to go that route as it was not adding any value. i wouldnt be opposed to go that route nowdays but there is a lot of shows in xem now and thats a lot of work to go update all those...
sven-7
commented
3 years ago Sounds like we have until October 1st now, per an email from today.
smoon1895
commented
3 years ago Sounds like we have until October 1st now, per an email from today.
Yes, thats correct. It will be October 1st, received the same email.
sven-7
commented
3 years ago Is the current plan just to stop supporting TVDB.....? If major library revamps are going to happen, that'd be good to know. Even some of the AniDB modes depend on TVDB for seasons.
Edit: A dev key can be applied for without paying for one, per some of their comments on their Git.
ZeroQI
commented
3 years ago Need to see if Plex TVDB proxy can be used passed the deadline It bothers me hightly to code to have TVDB V4 supported so they can be paid for the access.
Thinking about it but even TMDB series, how do we do the mapping?
If we lose the TVDB modes, me might as well make a separate agent for TMDB series and movies...
Cannot see how to remap TheTVDB to TMDB series...
reconman
commented
3 years ago If we lose the TVDB modes, me might as well make a separate agent for TMDB series and movies...
I personally would probably be fine with just using the built-in TMDB agent, I'm prioritizing TVDB metadata over AniDB right now, so the mapping doesn't really provide much benefit right now.
Edit: Now I see your point, the TMDB agent doesn't search in the movie part of the database if your anime library is based on TV Series.
reconman
commented
3 years ago Also, Plex employees stated that they don't care about breaking 3rd party plug-ins: https://forums.plex.tv/t/solved-dsmv7-how-to-update-plugins-agents-scanners/700715/15
So the future doesn't look too bright for custom agents anyways.
smoon1895
commented
3 years ago Need to see if Plex TVDB proxy can be used passed the deadline It bothers me hightly to code to have TVDB V4 supported so they can be paid for the access.
Thinking about it but even TMDB series, how do we do the mapping?
If we lose the TVDB modes, me might as well make a separate agent for TMDB series and movies...
Cannot see how to remap TheTVDB to TMDB series...
I would highy prefer it to be able to use TVDBv4 in the future. Just as an option for the users which are willing to pay for it. For me it would be a pain in the a.. to change all my metadata (got about 3000 Anime in my Plex).
Regarding TMDB: The problem here is that there are a lot of issues with the metadata like correct seasoning, titles etc., i tried it with a few of my series and it ended in a mess.
sven-7
commented
3 years ago I guess at this point, it's probably time to start seeing if I can make any native Plex agents work. Given Oct 1st is a month away...sigh. TVDB just had to ruin a good thing.
smoon1895
commented
3 years ago @ZeroQI are there any news on how it will go on with Hama? We have almost the middle of September and TVDBv4 will be starting in about 3 weeks and a half.
sven-7
commented
3 years ago @ZeroQI are there any news on how it will go on with Hama? We have almost the middle of September and TVDBv4 will be starting in about 3 weeks and a half.
I'm guessing at this point, it's going to be up to folks to figure things out on their own. With TheTVDB breaking things, things from HAMA and ASS are going to break as well. Pure AniDB via HAMA or moving to Plex's TVDB are likely the future options that exist. Not ideal, but if @ZeroQI does not want to support TheTVDB given their (poor) choices, then I'm guessing it's likely the end of an era.
ZeroQI
commented
3 years ago The issue is the mapping, without AniDB to TVDB mapping due to the need for users to pay for TheTVDB access, then it will be ANiDB OR TheMovieDb series, but not mapped to each other, and this need coding, not ideal...
If i spend quite some time to fix theTVDB to migrate v2 to v4 API, then TheTVDB will earn money over a user filled database, which i morally object to, and users will be paying them... Somehow i am annoyed i don't get a cut...
I am not using Plex anymore since few years, reading files directly from the nas, my NAS has been shutdown for month, and i busy finishing my house in France where i work from home currently, as i am short on funds to finish, so taking a month of free time to code that is definitely not in my priority, despite the impeding doom. Still thinking of hte way forward, but i have no way to know how used are tvdb modes, apart from tvdb4 pull requests....
sven-7
commented
3 years ago In theory, Anime-Lists could be updated to include TheMovieDB. TheMovieDB does have a field for External IDs which contains TheTVDB ID, so I imagine we might be able to query their database using TheTVDB ID and pull down TheMovieDB. This would create the three-way link, in theory.
Ultimately tho, it's understandable how you feel about TheTVDB. I don't like it either and was hoping a solution could be found. For now, I think I'm going to start the slow and painful process of moving my library over to Plex's native systems and figure out the best way forward.
reconman
commented
3 years ago My experience of the plex series agent:
I'm currently also trying out Jellyfin instead of Plex. Can't really say anything about it yet, I only installed it 2 days ago.
It has plugins for AniDB and Anilist, but of course no mapping function.
ZeroQI
commented
3 years ago Movies and series is only possible in Plex with ASS to my knowledge, and if Plex cancels scanners it's gone
TVDB modes is only ass/hama as well...
I need to know if Plex is heading towards removing scanner and agent support as the new library/agents hints which no longer support python
If so no point in developing short lived code... Wondering if Plex TVDB proxy could be used post October to keep functionality
smoon1895
commented
3 years ago Movies and series is only possible in Plex with ASS to my knowledge, and if Plex cancels scanners it's gone
TVDB modes is only ass/hama as well...
I need to know if Plex is heading towards removing scanner and agent support as the new library/agents hints which no longer support python
If so no point in developing short lived code... Wondering if Plex TVDB proxy could be used post October to keep functionality
Well, nothing is said about stopping the support for third party Scanners in Plex at the moment, i could also not remember me that there was said something about this.
Plex chose a way to continuesly use the TVDB API after the 1st of October this year in implementing an own Scanner and Metadata Agent directly in to Plex which is using multiple sources as source for the metadate.
What about implementing a feature in Hama to let the users decide if they want to use TVDB v4 by manually entering the own API-Key and PIN into the settings of Hama->TVDB and users which not have entered a Key and PIN can use Hama without TVDB?
With this you can let the users decide.
ZeroQI
commented
3 years ago Plex must be having a deal with TheTVDB unless the agent ask for an API key and pin code... Can you check? One could read the key in their agent to reuse it...
reconman
commented
3 years ago One could read the key in their agent to reuse it...
The TVDB agent still uses v2. The Plex TV agent is part of Plex binaries, so you'd either have to decompile the binaries or break open the HTTPS connection between Plex and TVDB.
smoon1895
commented
3 years ago Plex must be having a deal with TheTVDB unless the agent ask for an API key and pin code... Can you check? One could read the key in their agent to reuse it...
I think so ... otherwise I can´t explain why they are still be able to access thetvdb. As the Plex TV Series and Plex Movies Agent is a part of the Plex-Binaries i think you will not find out if they are using any API-Keys or however they are querying tvdb.
ZeroQI
commented
3 years ago Need the agent code, however it seems it won't be python anymore and could be compiled in binary, thus cannot load it unless I have a file location where it is stored...
ZeroQI
commented
3 years ago Retaining tvdb mode would be cool, so we could read or borrow TheTVDB API from Plex, since it is a Plex agent so not too diferrent from a standard tvdb library, or just access TheTVDB with no API searching the code in the web page (dirty!)... Unsure about Plex tvdb proxy, if we can use it to retain functionality by changing our URL to it in the meantime...
reconman
commented
2 years ago October 1st has passed, seems like the old API is still available?
thezoggy
commented
2 years ago correct, old api is sticking around for a bit per tvdb email last week
On that note: if you are still using any of our legacy APIs (v1 and v3) you can rest assured that they will not be deprecated until we feel that the development community has had sufficient time to migrate over to v4. However, these legacy APIs will be turned off in the first half of 2022, so we encourage you to reach out and create a v4 API key so that your project can receive all of the beneficial new data that the v4 API has to offer, and so we can address any questions you may have.
Roguyt
commented
2 years ago Is there any way to help the process to update Hama to either TVDBv4 or the migration to TMDB ? Losing Hama would be quite problematic and with such a vague deadline from tvdb it may come sonner than later.
notmutiny
commented
2 years ago It looks like someone on reddit asked the TVDB account how Plex agents like Hama should handle v4 licensing and it sounds like TVDB will approve Hama and anything else Plex related without needing a subscription if someone contacts them.
https://old.reddit.com/r/PleX/comments/iq2eud/thetvdb_user_subscriptions/hp9akop/
arabcoders
commented
2 years ago Also, it seems plex started to support their own database which is an aggregate from multiple databases imdb,tmdb,tvdb you can see it at https://watch.plex.tv/ , not sure if they provide support for external clients though.
They also support providing the old databases GUIDs
VegethB
commented
2 years ago It appears that plex is taking away plugin support altogether. What will happen to hama? In case, by staying at a PMS before the plugins are removed, will support continue? Emby and Jellyfin are not on the level of hama... especially with AniDB.
With hama I can use metadata from AniDB and images from other sources (which emby and jellyfin you have to manually enter the IDs and you are still limited to IDs only for TV Show).
arabcoders
commented
2 years ago It appears that plex is taking away plugin support altogether. What will happen to hama? In case, by staying at a PMS before the plugins are removed, will support continue? Emby and Jellyfin are not on the level of hama... especially with AniDB.
With hama I can use metadata from AniDB and images from other sources (which emby and jellyfin you have to manually enter the IDs and you are still limited to IDs only for TV Show).
Any sources for this ? i haven't been able to find any official word of plugins sunsetting completely, if it's comes to that, you can always switch to NFO scanner and add your data via tiny media manager or any other source including images etc.
ZeroQI
commented
2 years ago arabcoders
commented
2 years ago That's has been the case since 2018' afaik any plugins that interact with frontend stopped unless you use some other kind of viewer like kitana which exposes this UI. No news on completely shutting off plugins that deal with metadata and/or scanners, so far.
VegethB
commented
2 years ago It appears that plex is taking away plugin support altogether. What will happen to hama? In case, by staying at a PMS before the plugins are removed, will support continue? Emby and Jellyfin are not on the level of hama... especially with AniDB. With hama I can use metadata from AniDB and images from other sources (which emby and jellyfin you have to manually enter the IDs and you are still limited to IDs only for TV Show).
Any sources for this ? i haven't been able to find any official word of plugins sunsetting completely, if it's comes to that, you can always switch to NFO scanner and add your data via tiny media manager or any other source including images etc.
https://forums.plex.tv/t/new-plex-tv-series-agent-anime-library-pro-and-cons/798708/2
This post is to decide the general way to go, as far as mapping file format and metadata sources go...
HAMA is currently using TheTVDB v2 API, latest being v4 [https://app.swaggerhub.com/apis/thetvdb/tvdb-api_v_4] and the only one to be supported soon
https://www.facebook.com/TheTVDB/ posted the 16th of March
https://thetvdb.com/subscribe indicate two licensing modes — licensed - for companies, not suitable for open-source projects — End-User Subscriptions for 11.99$ [https://thetvdb.com/subscribe] or free by filling missing IMDB|EIDR IDs [no yet available] but i could create a key as developer for free
EIDR IDs [https://www.eidr.org/] are unique IDs that look nice in principle and would have been great as unique ID but never heard of them.
As a result
Plex had a TheTVDB proxy i may use or my agent could share their agent key (not sure it is good) or just work with their agent (hama secondary agent or not)
https://github.com/ShokoAnime/ShokoServer/issues/755 Shoko does support Plex as agent and scanner, but doubt it has HAMA flexibility for numbering and no other metadata system support movies and series in a single library (No even plex does normally)
I think TheMovieDB is the way to go since already supported, and supports both movies and series We could keep TheTVDB with API keys for end users that subscribe, but can't really get over the fact it takes user filled data to sell back to users, without giving free keys to uploaders...
Mapping
Ideally, AniDB (Anime movies/series) and MovieDb (anime/live movies/series) mapping makes perfect sense but would require a new database format on the same format as the current Anime-List but mapped to TheMovieDB instead, but that is some undertaking...