search

chibicitiberiu / ytsm

Self-hosted tool which manages your YouTube subscriptions, and downloads files automatically.
MIT License
316 stars 19 forks source link

Excessive number of API requests #68

Open leveled opened 5 years ago

leveled commented 5 years ago

YTSM seems to be generating an excessive number of API requests for me, to the point where I cannot add any additional subscriptions as my 10000 request/day limit is quickly reached. I have my synchronization schedule set as such

0 0 * * *

Which if I understand correctly should just be triggering once a day. I only have around 30-40 subscriptions, which doesn't seem like a lot. I'm not sure what is causing the high number of requests.

chibicitiberiu commented 5 years ago

I think I found the source of the bug in the synchronization code. During synchronization, one of the steps is to update the statistics of the videos (play count, ratings). The bug is at this exact line, which makes it reiterate videos more than once.

The line for video in itertools.chain(work_vids, self.__new_vids): should actually read for video in batch:.

I will submit a fix soon. If you don't care at all about these video statistics, you can modify this file manually and set _ENABLE_UPDATE_STATS = False.

chibicitiberiu commented 5 years ago

Thinking about it, I think that may not be the problem. In that for loop, there is no YtAPI call. The problem may simply be that it tries to update the statistics for a few thousand videos every day, which is the reason why you are reaching the limit.

Disabling that 'update stats' field will probably fix the issue, but you won't get any video statistics any more.

I think the solution would be to be more conservative about updating video statistics. For big channels, the general trend is that videos generate the most views in the first couple of days, after which views slow down. So I think updating statistics less frequently for older videos is probably the best approach.