[RFC] SSD Cache Management

[jonfake0]

Scope

Pre-load subset of library onto SSD for better performance

Long Form

I am planning on implementing this for my use case over the next few weeks but am willing to generalize this feature and make it OSS if others are interested.

Motivation

I like to skip around a lot when I watch videos and sometimes have to wait for it to buffer for a few seconds even on local network with Ethernet. I will add an m.2 SSD 800gb to my Synology where I will store a copy of a selection of videos from my library.

Secondary goal: Duplicate the cache onto a portable drive before traveling to places with bad internet

Functionality

I will describe how I use my library and how we can use the pattern of behavior to intelligently populate the cache. I have around 20 saved filters that I use. The videos are sorted randomly. I will watch a few and then go to a different filter etc. Most filters have the condition "play_count < 1" although some do not. If I run out of videos for a filter I will increment the count. But each filter has 100-1000 videos so I won't need to do that for a long time.

How I will populate the cache is to decide a number n_i for each filter_i that is the number of videos to load into the cache. I will add a tag to those videos to identify them as cached and then copy the files into the cache. Every night when I sleep a cron job will run that will find cached videos that have been watched, delete them from the cache (and remove the tag), and add new videos so that there are n_i unwatched videos for each filter_i.

Identified areas of risk

• few videos are unwatched for a filter and need to increment playcount condition
• Filters that don't have playcount condition. How do we determine that they have been watched and purge them from the cache? Maybe use most recently watched date to determine?
• Usage session during cache refresh.
• Wide range of file sizes for a single filter.
• Maintain parity between filters in ui and filters in cache system. If a named filter is modified, how do we ensure that the cache will be populated with the updated filter? Should the change apply to existing videos in the cache, or just videos entering the cache

Examples

Can provide some pseudo code if my intentions are unclear

Conclusion

Please discuss if this would be useful to OSS and build properly. Otherwise I will just implement with a python script. I could upload that to community plugin repo if there is interest in that

[feederbox826]

The implementation sounds like it would only get out of hand and doesn't exactly align with the core ethos of stash which is local content management. This would also have to expand it's own scope outside of stash itself because of the remote access, which would probably be better suited for a plugin. Notably

• management of an external library, with known duplicates
• validation and processing of that external library
• browser client must have knowledge of the external library and ability to access the remote library