search

damienhaynes / TraktRater

TraktRater is a tool written in C# to help users transfer user episode, show and movie user ratings and watchlists from multiple media database sites around the web.
623 stars 36 forks source link

Client doesn't respect API rate limits #148

Closed DavidGhetto closed 1 year ago

DavidGhetto commented 1 year ago

Trakt API has rate limiting that allows 1 POST call per second: https://trakt.docs.apiary.io/#introduction/rate-limiting

When I try to import a large IMDb list, it fails because it doesn't respect the limits and the client receives a 429 response:

2023-03-04 14:59:51.357 [INFO] [][01]: Importing page 1/25 IMDb custom list movies... 2023-03-04 14:59:51.358 [DEBG] [][03]: Address: https://api.trakt.tv/users/me/lists/24811994/items, Post: {"movies":[{"ids":{"imdb":"tt0000417"},"title":"Le voyage dans la lune","year":1902},{"ids":{"imdb":"tt0000439"},"title":"The Great Train Robbery","year":1903},{"ids":{"imdb":"tt0004972"},"title":"The Birth of a Nation","year":1915},{"ids":{"imdb":"tt0006206"},"title":"Les vampires","year":1915},{"ids":{"imdb":"tt0006864"},"title":"Intolerance: Love's Struggle Throughout the Ages","year":1916},{"ids":{"imdb":"tt0009968"},"title":"Broken Blossoms or The Yellow Man and the Girl","year":1919},{"ids":{"imdb":"tt0010323"},"title":"Das Cabinet des Dr. Caligari","year":1920},{"ids":{"imdb":"tt0011870"},"title":"Within Our Gates","year":1920},{"ids":{"imdb":"tt0011841"},"title":"Way Down East","year":1920},{"ids":{"imdb":"tt0012532"},"title":"Orphans of the Storm","year":1921},{"ids":{"imdb":"tt0012364"},"title":"Körkarlen","year":1921},{"ids":{"imdb":"tt0013626"},"title":"La souriante Madame Beudet","year":1923},{"ids":{"imdb":"tt0013086"},"title":"Dr. Mabuse, der Spieler","year":1922},{"ids":{"imdb":"tt0013427"},"title":"Nanook of the North","year":1922},{"ids":{"imdb":"tt0013442"},"title":"Nosferatu, eine Symphonie des Grauens","year":1922},{"ids":{"imdb":"tt0013257"},"title":"Häxan","year":1922},{"ids":{"imdb":"tt0013140"},"title":"Foolish Wives","year":1922},{"ids":{"imdb":"tt0014341"},"title":"Our Hospitality","year":1923},{"ids":{"imdb":"tt0014417"},"title":"La roue","year":1923},{"ids":{"imdb":"tt0015064"},"title":"Der letzte Mann","year":1924},{"ids":{"imdb":"tt0015324"},"title":"Sherlock Jr.","year":1924},{"ids":{"imdb":"tt1764657"},"title":"The Great White Silence","year":1922},{"ids":{"imdb":"tt0015881"},"title":"Greed","year":1924},{"ids":{"imdb":"tt0015400"},"title":"The Thief of Bagdad","year":1924},{"ids":{"imdb":"tt0015361"},"title":"Stachka","year":1925},{"ids":{"imdb":"tt0015772"},"title":"The Eagle","year":1925},{"ids":{"imdb":"tt0016220"},"title":"The Phantom of the Opera","year":1925},{"ids":{"imdb":"tt0015648"},"title":"Bronenosets Potemkin","year":1925},{"ids":{"imdb":"tt0015864"},"title":"The Gold Rush","year":1925},{"ids":{"imdb":"tt0015624"},"title":"The Big Parade","year":1925},{"ids":{"imdb":"tt0016332"},"title":"Seven Chances","year":1925},{"ids":{"imdb":"tt0015532"},"title":"Die Abenteuer des Prinzen Achmed","year":1926},{"ids":{"imdb":"tt0017925"},"title":"The General","year":1926},{"ids":{"imdb":"tt0017136"},"title":"Metropolis","year":1927},{"ids":{"imdb":"tt0018455"},"title":"Sunrise: A Song of Two Humans","year":1927},{"ids":{"imdb":"tt0018528"},"title":"The Unknown","year":1927},{"ids":{"imdb":"tt0018037"},"title":"The Jazz Singer","year":1927},{"ids":{"imdb":"tt0018192"},"title":"Napoléon vu par Abel Gance","year":1927},{"ids":{"imdb":"tt0018051"},"title":"The Kid Brother","year":1927},{"ids":{"imdb":"tt0018217"},"title":"Oktyabr","year":1927},{"ids":{"imdb":"tt0018806"},"title":"The Crowd","year":1928},{"ids":{"imdb":"tt0018839"},"title":"The Docks of New York","year":1928},{"ids":{"imdb":"tt0019254"},"title":"La passion de Jeanne d'Arc","year":1928},{"ids":{"imdb":"tt0019421"},"title":"Steamboat Bill, Jr.","year":1928},{"ids":{"imdb":"tt0019286"},"title":"Potomok Chingis-Khana","year":1928},{"ids":{"imdb":"tt0020286"},"title":"Prapancha Pash","year":1929},{"ids":{"imdb":"tt0020530"},"title":"Un chien andalou","year":1929},{"ids":{"imdb":"tt0019760"},"title":"Chelovek s kino-apparatom","year":1929},{"ids":{"imdb":"tt0018737"},"title":"Die Büchse der Pandora","year":1929},{"ids":{"imdb":"tt0019702"},"title":"Blackmail","year":1929}]} 2023-03-04 14:59:51.873 [DEBG] [][03]: Response: {"added":{"movies":50,"shows":0,"seasons":0,"episodes":0,"people":0},"existing":{"movies":0,"shows":0,"seasons":0,"episodes":0,"people":0},"not_found":{"movies":[],"shows":[],"seasons":[],"episodes":[],"people":[],"users":[]},"list":{"updated_at":"2023-03-04T13:59:51.000Z","item_count":50}} 2023-03-04 14:59:51.874 [INFO] [][01]: Importing page 2/25 IMDb custom list movies... 2023-03-04 14:59:51.874 [DEBG] [][03]: Address: https://api.trakt.tv/users/me/lists/24811994/items, Post: {"movies":[{"ids":{"imdb":"tt0020697"},"title":"Der blaue Engel","year":1930},{"ids":{"imdb":"tt0020629"},"title":"All Quiet on the Western Front","year":1930},{"ids":{"imdb":"tt0021577"},"title":"L'âge d'or","year":1930},{"ids":{"imdb":"tt0021571"},"title":"Zemlya","year":1930},{"ids":{"imdb":"tt0021079"},"title":"Little Caesar","year":1931},{"ids":{"imdb":"tt0022599"},"title":"À nous la liberté","year":1931},{"ids":{"imdb":"tt0022080"},"title":"Limite","year":1931},{"ids":{"imdb":"tt0022458"},"title":"Tabu: A Story of the South Seas","year":1931},{"ids":{"imdb":"tt0021749"},"title":"City Lights","year":1931},{"ids":{"imdb":"tt0021814"},"title":"Dracula","year":1931},{"ids":{"imdb":"tt0021884"},"title":"Frankenstein","year":1931},{"ids":{"imdb":"tt0022286"},"title":"The Public Enemy","year":1931},{"ids":{"imdb":"tt0022100"},"title":"M - Eine Stadt sucht einen Mörder","year":1931},{"ids":{"imdb":"tt0022150"},"title":"Le million","year":1931},{"ids":{"imdb":"tt0021739"},"title":"La chienne","year":1931},{"ids":{"imdb":"tt0023649"},"title":"Vampyr","year":1932},{"ids":{"imdb":"tt0023042"},"title":"I Am a Fugitive from a Chain Gang","year":1932},{"ids":{"imdb":"tt0022718"},"title":"Boudu sauvé des eaux","year":1932},{"ids":{"imdb":"tt0023158"},"title":"Love Me Tonight","year":1932},{"ids":{"imdb":"tt0023458"},"title":"Shanghai Express","year":1932},{"ids":{"imdb":"tt0023622"},"title":"Trouble in Paradise","year":1932},{"ids":{"imdb":"tt0023427"},"title":"Scarface","year":1932},{"ids":{"imdb":"tt0022913"},"title":"Freaks","year":1932},{"ids":{"imdb":"tt0023202"},"title":"Me and My Gal","year":1932},{"ids":{"imdb":"tt0024034"},"title":"42nd Street","year":1933},{"ids":{"imdb":"tt0024028"},"title":"Footlight Parade","year":1933},{"ids":{"imdb":"tt0023814"},"title":"The Bitter Tea of General Yen","year":1932},{"ids":{"imdb":"tt0024548"},"title":"She Done Him Wrong","year":1933},{"ids":{"imdb":"tt0023969"},"title":"Duck Soup","year":1933},{"ids":{"imdb":"tt0024803"},"title":"Zéro de conduite: Jeunes diables au collège","year":1933},{"ids":{"imdb":"tt0024069"},"title":"Gold Diggers of 1933","year":1933},{"ids":{"imdb":"tt0024216"},"title":"King Kong","year":1933},{"ids":{"imdb":"tt0023037"},"title":"Las Hurdes","year":1933},{"ids":{"imdb":"tt0024601"},"title":"Sons of the Desert","year":1933},{"ids":{"imdb":"tt0024481"},"title":"Queen Christina","year":1933},{"ids":{"imdb":"tt0025318"},"title":"It's a Gift","year":1934},{"ids":{"imdb":"tt0195256"},"title":"Shen nu","year":1934},{"ids":{"imdb":"tt0025316"},"title":"It Happened One Night","year":1934},{"ids":{"imdb":"tt0024844"},"title":"L'Atalante","year":1934},{"ids":{"imdb":"tt0024894"},"title":"The Black Cat","year":1934},{"ids":{"imdb":"tt0025878"},"title":"The Thin Man","year":1934},{"ids":{"imdb":"tt0025335"},"title":"Judge Priest","year":1934},{"ids":{"imdb":"tt0025913"},"title":"Triumph des Willens","year":1935},{"ids":{"imdb":"tt0026752"},"title":"Mutiny on the Bounty","year":1935},{"ids":{"imdb":"tt0026029"},"title":"The 39 Steps","year":1935},{"ids":{"imdb":"tt0026778"},"title":"A Night at the Opera","year":1935},{"ids":{"imdb":"tt0026138"},"title":"Bride of Frankenstein","year":1935},{"ids":{"imdb":"tt0026174"},"title":"Captain Blood","year":1935},{"ids":{"imdb":"tt0027125"},"title":"Top Hat","year":1935},{"ids":{"imdb":"tt0026866"},"title":"Peter Ibbetson","year":1935}]} 2023-03-04 14:59:52.012 [ERR ] [][03]: Response: Le serveur distant a retourné une erreur : (429) Too Many Requests. 2023-03-04 14:59:52.013 [ERR ] [][01]: Failed to send custom list items for IMDb movies

damienhaynes commented 1 year ago

Thanks for the report. Trakt must of introduced rate limiting on their API now so will need to add support to handle it. I'll look into it.

damienhaynes commented 1 year ago

Made a quick retry method to handle this event. You should now see something like this in the log and a retry after X seconds:

2023-03-05 11:54:14.677 [INFO] [][01]: [9/9] Removing all episodes of Game of Thrones from trakt.tv watched history
2023-03-05 11:54:14.677 [DEBG] [][03]: Address: https://api.trakt.tv/sync/history/remove, Post: {"shows":[{"ids":{"trakt":1390},"title":null,"year":null}]}

2023-03-05 11:54:14.936 [ERR ] [][03]: Response: The remote server returned an error: (429) Too Many Requests. {"name":"AUTHED_API_POST_LIMIT","period":1,"limit":1,"remaining":0,"until":"2023-03-05T00:54:18Z"}. Waiting 1 seconds before trying again.

2023-03-05 11:54:15.937 [DEBG] [][03]: Address: https://api.trakt.tv/sync/history/remove, Post: {"shows":[{"ids":{"trakt":1390},"title":null,"year":null}]}
2023-03-05 11:54:16.293 [DEBG] [][03]: Response: {"deleted":{"movies":0,"episodes":1},"not_found":{"movies":[],"shows":[],"seasons":[],"episodes":[],"people":[],"users":[]}}