eclipse-archived / smarthome

Eclipse SmartHome™ project
https://www.eclipse.org/smarthome/
Eclipse Public License 2.0
862 stars 783 forks source link

[core / cache] File based cache service #6041

Open cweitkamp opened 6 years ago

cweitkamp commented 6 years ago

I saw a lot of implementations of a file based cache service - specially inside the TTS add-ons (see Google TTS or Polly TTS). I think we should generalize those and port the feature to ESH to make it available as public API. We could save a lot of code in the future and streamline the usage throughout the whole framework.

To clarify: I am not talking about a TTS cache only. I can think of some other use cases which can benefit from such a feature. E.g. the weather bindings for downloading weather icons or media bindings for downloading thumbnails, cover images and so on. I myself implemented approaches in the new OpenWeatherMap binding or in the Kodi binding based on an ExpiringCache. But that seems to be a bad idea for two reasons. First of all storing something in an ExpiringCache will raise the memory usage. This is a big disadvantage on embedded devices. Second it will not survive a restart.

I opened this issue in ESH to share my thoughts and start a discussion / brainstorming on it.

wborn commented 6 years ago

It would also be nice if it were possible to limit the disk space used for caching and define what is removed from the cache(s) and when.

maggu2810 commented 6 years ago

@cweitkamp Do you consider to use one caching API for different cache storage implementation?

E.g. a product for an embedded device that would like to reduce the write cycles to its persistent storage (flash / SD-Card / ...) but "enough" RAM can use the "volatile (expiring) cache" implementation and a product for an embedded device with little RAM but that don't care about write cycles can use the "persistent (expiring) cache" implementation.

This way the code that would like to cache something does not need to know -- and the developer does not need to decide -- which cache implementation should be used...

Perhaps we could also design one cache that has a default storage strategy (volatile or persistent) and can be configured to use specific strategies for specific "namespaces".

cweitkamp commented 5 years ago

@wborn @maggu2810 Thank you for your initial comments and ideas.

I have started to implement a util class for the OWM binding called ByteArrayFileCache. I want to use this implementation in the Kodi binding too. This util class handles basic operations for e.g. reading and writing (image) files and clearing the folder. It ensures we have unique file names inside the folder.

The current implementation creates a new sub directory for each service - specified by the service PID - inside a cache/ folder in the ConfigConstants.getUserDataFolder(). This should be the default behavior except the user defines a specific path to a different location like @maggu2810 suggested.

Additionally this implementation is limited to read / write byte[] data. We of course should make this more unspecific but I could not figure out a good way for an architectural interface structure.

A possible interface for different cache services.

I am looking forward for your feedback.