jvilk / BrowserFS

BrowserFS is an in-browser filesystem that emulates the Node JS filesystem API and supports storing and retrieving files from various backends.
Other
3.06k stars 216 forks source link

Separate backends #369

Closed james-pre closed 8 months ago

james-pre commented 9 months ago

Backends should be separated into different NPM packages so users don't have to import needed backends:

NPM package Description
@browserfs/core Includes the core code for BrowserFS and these backends: InMemory, MountableFileSystem, OverlayFS, AsyncMirror, FolderAdapter
@browserfs/fetch Fetch/HTTPRequest/XMLHTTPRequest backend
@browserfs/dom Backends which require DOM APIs (i.e. LocalStorage and IndexedDB)
@browserfs/emscripten Emscripten backend
@browserfs/zip ZipFS backend (will be updated to include explode, unreduce, and unshrink)
@browserfs/iso IsoFS backend
@browserfs/dropbox Dropbox backend

Note that ownership of the @browserfs organization on NPM is pending and this would resolve #233 and #200.

zardoy commented 4 months ago

@james-pre Hey! I'm sorry to ask you again, to where I can find a new repo for the remote backend (dropbox)? I want to implement google drive backend as I really need it. If there is anything I can do for you please let me know. thx also waiting for npm packages to be published!

james-pre commented 4 months ago

@zardoy,

I made the decision not to include any backends for 3rd party services since it was already a huge amount of work splitting up the codebase, rewriting the internal API to use promises, and writing the compatibility for fs promises. Once BFS is up to Node 20 compatibility, I would work on other backends.

If you want, I would love a PR for the Dropbox backend that brings it up to the current BFS internal API.

Thanks.

zardoy commented 3 months ago

@james-pre

I made the decision not to include any backends for 3rd party

I think it would be enough to just include some generic blackened that would be easy to extend so you can just fill the required implementation for operations like reading directories and files. Though I believe these 3rd party backends might be quite popular I think it would be better to have official support for them.

Once BFS is up to Node 20 compatibility, I would work on other backends.

Have any roadmap or timetable? I'm looking forward to this happening!

If you want, I would love a PR for the Dropbox backend that brings it up to the current BFS internal API.

Sorry I don't need Dropbox backend, but here I implemented a Google Drive backend: https://github.com/zardoy/browserfs/blob/google-drive/src/backends/GoogleDrive.ts (and already tested a bit)

also btw I wanted to support old Node.js API so I added back callback-style support in API

EDIT: sorry for the delay, didn't have much time to finish the Google Drive backend, now I need to cleanup all that somehow :)

james-pre commented 3 months ago

I agree with you, I just haven't implemented the 3rd party backends.

As for your callback based backend, the backends must implement the internal BFS API (using promises). The compatibility/emulation part of the code then adds the full Node callback API, among other things.

I don't have any exact times for when I want to get things done, since I work on BFS in my free time (which can vary widely).

Thank you for all the interest!

james-pre commented 3 months ago

@zardoy,

I think BFS/core has gotten to a really good point for me to look into implementing the 3rd party backends. First, I want to overhaul the other packages (DOM, Fetch, Worker, Zip, Iso, etc.).

Maybe I can look into implementing them in the next month or so?

zardoy commented 3 months ago

IMO it would be much better to support all core features first. For example I'm still eagerly waiting for latest Node.js to be supported (for some things I personally do workarounds like https://github.com/zardoy/prismarine-web-client/blob/f16dbdd61f5b524c892d3da9bfa78f5b18ed13d8/src/browserfs.ts#L33) On the other hand I'm really interested in zipfs as this the the backend I use most.

zardoy commented 3 months ago

Maybe I can look into implementing them in the next month or so?

Anyway It would so cool! Looking forward to it!