Trakt updated shows API is now hard limited to 100 results per page meaning shows aren't updating

[Js41637]

Not sure when this change happened but I noticed recently that the calendar hasn't been updating, this issue #1272 is also probably related.

When DuckieTV checks for updated shows it calls https://api-v2launch.trakt.tv/shows/updates/{date}?limit=10000, since this was added it there was never a limit on the amount of results that could be returned but it is now capped at 100 which means there is basically 0 chance any of your shows will be getting updated.

Changing the update period also isn't likely to help as the changes we request are only yyyy-mm-dd which means you'll only be getting changes for the whole day so even if you check every hour, you'll still only be getting all changes for that day and if there is already over 100 then you're not going to get any updates.

Only way to fix this is the code is going to have to change to load every page of changes while the total results returned is less than 100, unless there is another way to get changes.

Only work around for now is to go into Settings > Backup > Refresh All Series

Unforunately this also means all old versions of DuckieTV now suffer this issue.

[Js41637]

fanart.tv is still garbage, maybe if I can be bothered i'll change it to use TMDB, the quality of the images and the API is much better. Also TMDB has recently completely removed limits from their API so it should be very easy to do this.

[ghost]

fanart.tv is still garbage, maybe if I can be bothered i'll change it to use TMDB, the quality of the images and the API is much better. Also TMDB has recently completely removed limits from their API so it should be very easy to do this.

This is a much needed improvement.

[ilike2burnthing]

Something I've managed to see now that the console is a lot clearer is that I'm getting a few 404 errors: https://api-v2launch.trakt.tv/shows/147324?extended=full https://api-v2launch.trakt.tv/shows/152399?extended=full

Is there any way to track down which shows they refer to? I'm assuming they were some early announcements I had added but have been changed.

EDIT: 152399 - Loki

[garfield69]

• open the development console with F12
• and click on the Console tab
• at the > paste the following CRUD.FindOne('Serie', {TRAKT_ID: 147324}).then(function(result) {console.log(result.name)})
• and press enter.

if the record was found you should see the name of the series .

[ilike2burnthing]

Marvel's Helstrom. Thanks!

[garfield69]

I'm not seeing the db-records popup on the Dtv main page while the update is progressing, just me?

[Js41637]

Probably because its based on the amount of writes that are happening, because it doesnt spam 1000 api requests at once now it never reaches the write threshold to show the popup.

ideally we redeisgn it to show when trakt is updating and its progress.

[ilike2burnthing]

Yea, I've only had it pop up for a split second maybe twice in the last few days.

[Js41637]

I have updated the query monitor to now show the current progress of trakt updates. It also requires a new translation.

[garfield69]

want me to do the translations?

[Js41637]

Yes please.

On Thu, 5 Nov 2020, 5:52 pm garfield69, notifications@github.com wrote:

want me to do the translations?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/SchizoDuckie/DuckieTV/issues/1273#issuecomment-722181374, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABPX4JIBFOM4WNM5DZLIQF3SOJDS7ANCNFSM4ROGMRQQ .

[garfield69]

translations done. For those that want to try it out, my personal build link for the latest win64 is in https://github.com/SchizoDuckie/DuckieTV/issues/1273#issuecomment-720350100

[ilike2burnthing]

Works well :)

[roshavagarga]

Can confirm that it works pretty well here too, thanks for the personal build @garfield69 :)

[AlienProber]

I finally gave up on Duckie and have installed and setup Sonarr.

[jagre77]

Yes, your personal build is working perfectly on my end. Haven't spotted any issues yet (and thanks for the EZTV fix).

[VulcanTourist]

here is my personal build for windows x64 of the latest Nightly using the source from the DuckieTV github. https://www.dropbox.com/s/dyo3ty06fjcnjnq/DuckieTV-202011071931-windows-x64.zip?dl=1 [edit4] added latest Dtv PB for win64 with SE domains managed by SettingsService (via console not UI for now).

Using this version changes the behavior of image/banner retrieval for the Favorites view. Now, instead of seemingly random retrieval of some but not all each time, it seems to retrieve all that are available up to a certain point and then no more (for me it stopped in the "S"es), even when an image is available and appears in the series Details. That is what I saw the first time that I clicked into Favorites with this new version. When I later returned to Favorites, however, it had quietly filled-in the remainder for ALL of them (well, all that existed). This is the FIRST time ever that I have seen Favorites images for virtually every series in the list.

Nothing I can say yet about the refresh issue, as I don't know how to effectively test it.

[garfield69]

Yeah, the first time you load up the favourites page, Dtv floods the FanArt site with requests for pictures. But the site cannot handle the load, and at some point is starts rejecting requests with 503s. There's been talk about switching to IMDB but I don't know if that will ever happen. But I think I'll have a look at trying to pace out the http GETs of the pictures, and see if I can get it to keep the server happy.

[garfield69]

Nah, this http poster fetch from fanart is too deep for me to get a handle on. On a more up beat note, I've managed to get my duckietv-builder to generate packages for windows,debian,linux,osx and the zips for the nightly chrome extensions (action mode and tab mode). see https://github.com/SchizoDuckie/DuckieTV/issues/1273#issuecomment-720350100 for share links.

[AlienProber]

Don't learn do you? TheTVDB and IMDB are not your friends. TheMovieBD and others are.

[garfield69]

@Js41637 so I guess the next step is to also apply this fetching process to the backup import otherwise 429 rate limited hits will prevent anyone successfully restoring a backup ?

[garfield69]

fielded several reddit users that, in a mistaken belief there was something wrong with their Dtv, wiped their old Dtv and installed a fresh one, then tried to restore from a Dtv backup. Which failed due to the Trakt.tv limiter. @Js41637 are you available to apply the same limiter handling logic to the Dtv Import code, as possibly the Trakt.tv import code ?

[garfield69]

Hah. Seeing multiple (23 of 317) errors Failed to load resource: the server responded with a status of 502 () during my trakt.tv update cycle this morning. Looks like the trakt.tv servers in my region are struggling to keep up, even with the rate limiting the have imposed ;-D Hope they are not under a DDoS attack, that could really complicate things ;-(

[ilike2burnthing]

Thankfully just the one for me.

https://api.trakt.tv/shows/134421?extended=full - working https://api.trakt.tv/shows/134421/seasons?extended=full - working https://api.trakt.tv/shows/134421/seasons/1/episodes?extended=full - 502 https://api.trakt.tv/shows/134421/seasons/2/episodes?extended=full - 502

Same issue if I try to refresh the series (The Rookie) manually.

[VulcanTourist]

Why aren't the results from fanart being stored persistently? The fetching process need only be done to replace changed images or obtain new ones. For missing ones the site query would be done immediately, but general update checks could be scheduled for perhaps once a month and staggered to reduce load.

On Sun, Nov 8, 2020, 8:15 PM garfield69 notifications@github.com wrote:

Yeah, the first time you load up the favourites page, Dtv floods the FanArt site with requests for pictures. But the site cannot handle the load, and at some point is starts rejecting requests with 503s. There's been talk about switching to IMDB but I don't know if that will ever happen. But I think I'll have a look at trying to pace out the http GETs of the pictures, and see if I can get it to keep the server happy.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/SchizoDuckie/DuckieTV/issues/1273#issuecomment-723743700, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABOIV7XWKIGKUR4EY5ZABL3SO5UHXANCNFSM4ROGMRQQ .

[garfield69]

Dtv does not save the images themselves (that would take too much room in a limited sized sandboxed dataspace defined by chromium) So Dtv saves the image URLs in the DB instead. When it comes time to display the images, that is when the urls are used to fetch the images. And that is when the FanArt server gets overloaded.

[VulcanTourist]

That seems like a horrid limitation! It's quite literally wrecking this one aspect of DuckieTV's functionality. This image retrieval almost never works fully for me. I know I said earlier that I thought it had improved, but it was just one very lucky instance. I'm now back to having about half the images at best.

Isn't there some means of working around this limitation, other than ditching Chromium as a rendering engine?

On Sun, Nov 15, 2020, 11:01 AM garfield69 notifications@github.com wrote:

Dtv does not save the images themselves (that would take too much room in a limited sized sandboxed dataspace defined by chromium) So Dtv saves the image URLs in the DB instead. When it comes time to display the images, that is when the urls are used to fetch the images. And that is when the FanArt server gets overloaded.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/SchizoDuckie/DuckieTV/issues/1273#issuecomment-727619498, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABOIV7XF6OE3K2SGV7GAGOTSQAQSNANCNFSM4ROGMRQQ .

[garfield69]

No. Chromium is the internal engine that both the Chrome Browser and NWJS use. This discussion is off topic for this ticket.

[garfield69]

The latest personal builds which include FanArt fetching improvements are available at the usual comment at https://github.com/SchizoDuckie/DuckieTV/issues/1273#issuecomment-720350100

[Js41637]

Trakt has made some changes to its changes endpoints in response to my issue and recommendations, hopefully we can change to using a more optimized system to keep data updated.

[lgmorand]

I'm using the late nightly release 20201120, and I still have to manually update each show. did I miss something ? when is the refresh triggered ?

ps: thanks all for the great work

[garfield69]

The default setting is 12 hours. You can increase the frequency of updates by navigating Dtv>Settings>Trakt.tv>Trakt.tvUpdatePeriod and changing the hourly rate. 1 hour is the smallest delay.

If you are on a macOS do not forget to drag the DuckieTV app from your Applications folder to the trash bin before running the Dtv Nightly Installer. The installer silently fails to upgrade your Dtv if the old App is present.

[lgmorand]

thanks ! I just installed it and thought it would be on startup. I'll let it turn few days to check

[ianwday]

Is there a limit of number of series you can have in DuckieTV because of this API change? In my About DuckieTV v202101052304 section, it says DB Series 1437 but, whenever DuckieTV updates, the total number of shows it's updating is always 1000. As a result, series that I've recently added are not on the calendar. I can add the show as a favourite but when I close and re-open DuckieTV, it is gone.

[forthehorde100]

I am having this bug on my duckieTV app. Refresh does not solve the problem either. about half of the shows i'm following just stop showing after a few episodes.

[VulcanTourist]

I am having this bug on my duckieTV app. Refresh does not solve the problem either. about half of the shows i'm following just stop showing after a few episodes.

Of course refresh does not resolve the issue. Have you read the entire conversation and applied the solution that others have used?