Add option to not use SQLite and default to that

[markg85]

🚀 Feature Requests

Contextualize the feature

? Not even sure what the question means.

Describe the feature

I was profiling a long task that opens a couple thousand small cache files. In this profile i was surprised to see calls going back to SQLite take up ~10% of all the time. Why? Why is SQLite in the cache path at all??

I frankly don't even care much about the "why" part here. The cache is a key -> file content as value type store with an URL as additional option (dependent on which API function you call) which downloads the file that needs to be cached if it's not cached yet.

When you call a cache function (like getSingleFile) then you already provide the URL the cache needs to download. It should do all other logic without SQLite, right? For example, it can hash the url and use that as key. It can store a file by that hash in the temporary app file store and call it done. Getting the cached data is as simple as doing the hash of the url again, testing if the file with that hash exists and return it's content if it does.

I'm hoping for an option in the config to disable SQLite completely and enable that option by default. Granted, this is from a very narrow point of view with just a file being cached and retrieving it. Perhaps there are other features that do require a database?

Platforms affected (mark all that apply)

Irrelevant, platform agnostic.

• [ ] :iphone: iOS
• [ ] :robot: Android

[markg85]

To amend. I just tried a simple novice "cache" mechanism that isn't much more then this:

  static Future<File> getImage(String url) async {
    Directory cachePath = await getTemporaryDirectory();

    String hashedUrl = blake3Hex(url.codeUnits);
    String composedPath = "${cachePath.path}/${hashedUrl}";

    if (!File(composedPath).existsSync()) {
      final response = await http.get(Uri.parse(url));
      await File(composedPath).writeAsBytes(response.bodyBytes);
    }

    return File(composedPath);
  }

I am aware that i missed error checking here. It's far from a complete solution but is enough to play with.

Using this as cache is far faster then flutter_cache_manager. Please look at your cache implementation and see if you can try and make it better. I'm guessing a vast amount of people using this library use this for image caching so optimizing for that usecase could perhaps be worth it?

[mvanbeusekom]

Hi @markg85,

I am not to familiar with the internals of the cache_manager package, however I see that it is possible to use multiple storage options.

According to the "How are cached files stored" FAQ question in the README:

Information about the files is stored in a database using sqflite on Android, iOS and macOs, or in a plain JSON file on other platforms. The file name of the database is the key of the cacheManager, that's why that has to be unique.

So this means files are indeed stored on the file system, but additional meta data is stored in the SQLite database. Meta data of cached objects consists out of:

• URL from where the file was downloaded.
• Path where the file is stored on disk.
• Expiry date time.
• Size of the file.
• Optional eTag provided by the server for cache expiry.
• Date time indicating when file was last used.

Since the cache_manager needs to store additional meta data along with the file to cache, simply using the file system is not enough. A solution might be to use something like "shared_preferences" to store this information. And it is entirely possible to create a new package that replaced the SQFLite store. As an example you can have a look at the implementation of the flutter_cache_manager_firebase package, which replaces SQFLite with a Firebase implementation. We would be happy to merge an PRs.

[markg85]

I see, while it's a general purpose cache, it has additional features that make it a bit heavier. Fair enough.

Still, a simple version build for just caching and be as efficient in that as possible would be welcome! The difference i'm observing with my novice cache vs flutter_cache_manager are so substantial that i consider dropping the latter. I also don't have a use for it's features that would've made it more attractive.

I'm also curious how creating a new package in this mindset (on top of flutter_cache_manager) is any more efficient? Sure, it might faster! But from an application APK point of view you'd drag in yet more dependencies and grow in size, no? It seems wasteful to me but that's just a gut feeling.