Option to download only new - not whole channel/playlist?

[TekWarren]

This looks super clean and was easy to get setup. The only thing I was wishing for was an option to choose only NEW videos from a channel vs all.

If I had to add anything else it would be the option to specify a time for the index also. For example, checking/downloading at a specific time.

Very awesome project, high hopes for you!

[kieraneglin]

Thank you very much! That's a fair shake, I'll add it to the list.

I'm currently working on a mechanism to greatly speed up indexing in certain cases. I'll let you know once that's pushed because it might be an alternative solution for the indexing time feature.

Thank you for the feedback! 🙌🏻

[kieraneglin]

To elaborate, indexing at a specific time is difficult in the cases where the time it takes to index is greater than the period between index runs. More clearly, if you have it set to run at midnight each night but indexing takes 26 hours, that's a problem. Some similar apps run to this issue so I intentionally took a slightly different approach with index scheduling to avoid this.

This is also coupled by the fact that only so many indexing events can run at a time before Google starts throttling or rejecting your requests. The app does gracefully handle failures, but that actually can make this stampede problem worse in some cases. If somebody were to schedule all of their indexing events to happen at midnight they would have to be staggered (which makes the first problem more likely). The app does currently only run a certain number of indexes concurrently to try and avoid rate limiting and I'll be tweaking this as time goes on.

There are definitely ways around it if that's what it comes to, but I'm hoping the fast indexing approach will make this problem mostly irrelevant 🤞🏻

[TekWarren]

Appreciate the reply and open-ness to suggestions. Makes sense about the indexing. Look forward to the possibility of the "only new" videos from a channel versus ALL.

Also had a thought for my timing based scenario. Perhaps the docker container itself could be scheduled to be "up" only certain hours of the day.

[kieraneglin]

@TekWarren Done! I've Added the ability to specify a cutoff date for a Source where only videos created on or after that date will be downloaded. To get the behaviour you want you can set it to the current day but you can also set it to any other day past or future (not sure why you'd want future but I'm not here to judge). This shouldn't act weird for existing media/Sources, but it won't delete any old media or anything. You'll need to add a new Source to get the full effect.

I've also implemented fast indexing for Media Profiles. This performs an initial "slow" index, then schedules followup slow indexes monthly. In the meantime, it runs a very fast and lightweight index every 15 minutes using an RSS feed and updates/downloads based on that. There's a whole spiel on the Media Profile form that explains it well.

I've pushed the release and it should be available on Unraid within ~2 hours, but you can force update the container if you want it sooner. With that I'm going to close this issue, but please open a new one if you see fit!

Thanks again for the feedback 🤙