Implement `file_system` for the web using IndexedDB to store the files

[sidrao2006]

🏗 Enhancement Proposal & Pitch

Currently, file_system's web implementation uses MemoryFileSystem. Migrating it to use IndexedDB would enable true web caching and make the implementation native.

Platforms affected

🌐 Web

[sidrao2006]

@renefloor , I am able to create something that would act as a 'file' but there's no way I can return an instance of File using indexedDB. So, do you want to let this use MemoryFileSystem for the time being or implement a custom variant of File which uses io.File on all platforms except for the web for which we'll implement all the methods using indexedDB?

[renefloor]

I think it would be nice if we can migrate to cross_file

[sidrao2006]

@renefloor , so, the way cross_file on web works is that it needs a url to which it saves the file.. it's not easy to do this in indexedDB..I'd have to implement it something like this.

cross_file's default implementation, where it auto-generates the path, doesn't really save the file at all. The 'file' is destroyed as soon as the webpage is closed

[sidrao2006]

The best solution I can think of is to extend and customize the saveTo method of XFile for the web. But we'd also have to disable the XFile.fromData constructor since it would generate a temporary url and mess up the indexedDB

[sidrao2006]

I hope you've considered the degree of this change..cross_file supports much fewer operations and properties than the existing file package's implementation.

[aytunch]

@sidrao2006 thanks for this issue. We are working on a video intensive web app and we want to preload the next couple of videos in the feed. However let alone caching, even downloading in the web platform seems not possible if I am not wrong. Can you think of a way to use flutter_cache_manager to save videos from a network url?

[sidrao2006]

@aytunch, that doesn't seem right. This package is definitely able to download files across all platforms, including the Web. As for caching, a 'memory file system' is used in the Web implementation. This approach technically doesn't cache any content, but it temporarily behaves that way.

The package is already being used to cache images too, and I can't see why it won't work with videos. Am I missing something? cc @renefloor

[aytunch]

@sidrao2006 This is from the video_player package:

"The Web platform does not suppport dart:io, so avoid using the VideoPlayerController.file constructor for the plugin. Using the constructor attempts to create a VideoPlayerController.file that will throw an UnimplementedError."

This was what made me think files could not be saved in a temporary location in Web. I guess according to you comment above, we can save to a temp location but the video_player can not read from there right?

[sidrao2006]

An implementation of FileSystem that exists entirely in memory with an internal representation loosely based on the Filesystem Hierarchy Standard. This implementation of the FileSystem interface does not directly use any dart:io APIs; it merely uses the library's enum values and interfaces. As such, it is suitable for use in the browser.

Flutter API Docs - MemoryFileSystem

[renefloor]

If we have to extend and override such crucial features we should not use cross_file I think. We might need to create our own implementation of File for web. It's just something I don't really like, that every package has their own version of 'file'. This is a broader Flutter issue and not something we can easily solve.

[sidrao2006]

I agree that there should have been a cross platform, universally accepted implementation of file but it's rather difficult to achieve this solely because there's no standard file system on the Web. For some applications, Indexeddb may be the best and for others, probably the Cache API.

Right now, using cross_file as a base is our best bet. I could try to implement a dart:io style file system interface but since it deals with too many operations and provides a large number of methods and properties, it could be a bit unreliable / unpredictable if I implement it using Indexeddb

[sidrao2006]

@renefloor , how would you like me to proceed with this? ⬆️

[renefloor]

Sorry I'm a bit busy and this is not my highest priority at the moment. You talk about using IndexedDB to store the cache. I can understand to use that to store the cache information (which files we have and how long they are valid), however, how do you want to store the files themselves?

[sidrao2006]

I understand. My plan is to store the files as globs in the indexedDB. I did look into the Cache API and other storage mechanisms but they either have poor cross browser support or are not easy to work with. I'd be open to suggestions though.

[muriloventuroso]

Any news on this?

[kgamecarter]

Why not use cache storage api ?