search

screeny05 / letterboxd-list-radarr

Connect radarr to letterboxd.com lists
https://letterboxd-list-radarr.onrender.com
MIT License
285 stars 27 forks source link

https://letterboxd.com/films/popular/ #17

Closed azumukupoe closed 1 year ago

azumukupoe commented 3 years ago

Could you make it work?

screeny05 commented 3 years ago

I'm afraid i cannot fetch the whole list, as it contains over half a million movies. That would probably take forever. I could make the list work with a hard pagecount-limit, so you could fetch the first 50 pages or something.

Calculating the runtime for /films/popular is pretty easy:

Accumulated response-time: ((564035 + 7834) * 200) = 114373800ms ~= 32h

If you can run 10 requests in parallel, that would get down to maybe 5h.

But i'm afraid that something along the way will get broken. The http-request might timeout, your redis might eat all of your memory, radarr will be unable to handle such a large amount of movies. This is not really feasible.

azumukupoe commented 3 years ago

Maybe fetch only the first 5 pages?

screeny05 commented 3 years ago

That would be possible. Maybe let the User adjust the limit with an optional parameter.

nathanbirrell commented 1 year ago

Is there a limit parameter, or a perPage parameter like in the API?

I would be happy with just the top 5 results from https://letterboxd.com/films/popular/this/week/

screeny05 commented 1 year ago

Can now be done. I've added a limit-parameter so you can fetch only the amount of movies you want to.

But there's a catch: letterboxd.com only allows us to fetch /films/popular/. Nothing else. No filters, no time, no genre, etc. See https://letterboxd.com/robots.txt