Adding Bulk operations for `readFile`, `writeFile`, and `unlink`.

[tiagobento]

Hi all!

Recently we've been experimenting with LightningFS and isomorphic-git on my team and we've got pretty interesting results with the combination of the two libraries. Thanks for creating and maintaining these two awesome projects!

As we've started to refine our solution (basically, it's an environment where you can upload folders, clone GitHub Gists, or even Git repositories and edit the files with rich graphical editors), we started to feel that big folders/repositories were giving LightningFS/isomorphic-git a hard time.

Profiling our application, I could notice that the writeFile operations were being done with some sort of throttling. I later discovered is really the case, as you mention at this very project's README file.

writeFile, readFile, and unlink are throttled by IndexedDB

After looking around and reading about IndexedDB, I came across this interesting StackOverflow answer, stating the Dexie could realize bulk operations. That's when I started to deep-dive into this repository's code to see if I could leverage Dexie's capability of bulk-writing.

To my (very positive!) surprise, I could simply replace the IdbBackend.js with an implementation that uses Dexie and add some new methods to the API.

I'm really happy with the results so far, and I see great potential in leveraging these bulk operations in isomorphic-git as well.

Results

My tests involved:

1. Uploading a folder with 1442 files.
2. Saving all files to IndexedDB using LightningFS (create files part on the images)
3. Running a git init using isomorphic-git
4. Running a git add . using isomorphic-git
5. Running a git commit using isomorphic-git

Before:

await Promise.all(files.map((f) => storageService.createFile(f)));

After:

await storageService.createFiles(files);

NOTE: storageService there is just an abstraction of our application around LightningFS. No big deal, you can see the code here: https://github.com/caponetto/kogito-tooling/blob/KOGITO-5074-with-settings/packages/online-editor/src/workspace/services/StorageService.tsx#L66-L90

In this PR

I didn't want to break anything, so what I did was I created a new idbBackend implementation called DexieBackend and made the DefaultBackend constructor accept a delegate as a parameter. People wanting to use this implementation can easily configure as you can see on src/__tests__/fs-dexie.promises.bulk.spec.js.

More than that, I added three new methods: writeFileBulk, readFileBulk, and unlinkBulk. These are optional and will throw an error if you don't provide an idbBackend implementation that has them.

To be able to run the tests with this new DexieBackend implementation, I added dexie as a devDependency. Also, the peerDependency there is for people consuming this library to know the version that's being used in tests.

Anyway, hope you find this contribution useful and that this PR can make into this library. I already started experimenting using these bulk operations in isomorphic-git, let's hope I get some interesting results soon :)

Looking forward to hearing your feedback!

[tiagobento]

Sorry about the big changes on package-lock.json, looks like I'm using a version of npm that decided to change the format of the file. Can you tell me which version you're using so I can fix this? :)

[tiagobento]

Any feedback?

[jcubic]

Hi, just found this PR. Sorry, but I don't see the place for this change. LightningFS is a library that is a replacement for the NodeJS fs module, that doesn't have such functions.