jvilk / BrowserFS

BrowserFS is an in-browser filesystem that emulates the Node JS filesystem API and supports storing and retrieving files from various backends.
Other
3.06k stars 216 forks source link

Use in Vite #335

Closed meponderR closed 1 year ago

meponderR commented 1 year ago

With many people migrating away from Webpack and migrating to alternatives, an example of how to use BFS in Vite similar to the ones for Browserify and Webpack would be helpful. I have not been able to figure out how to use BFS in Vite.

meponderR commented 1 year ago

I have figured out how to implement it. Here is an example that is similar to those found in the README. Thanks #304!

//vite.config.js

import { defineConfig } from "vite";
import inject from "@rollup/plugin-inject";
import { readFileSync } from "fs";

// https://vitejs.dev/config/
export default defineConfig({
    plugins: [
        inject({
            BrowserFS: "/src/bfs.js",
            process: "/src/bfsProcess.js",
        }),
    ],
    // global process from browserfs
    define: {
        global: {},
    },

    resolve: {
        alias: {
            buffer: "browserfs/dist/shims/buffer",
            fs: "browserfs/dist/shims/fs",
            path: "browserfs/dist/shims/path",
            processGlobal: "browserfs/dist/shims/process.js",
            bufferGlobal: "browserfs/dist/shims/bufferGlobal.js",
            bfsGlobal: "browserfs",
        },
    },
});
//src/bfs.js
import * as bfsFS from "browserfs";

export { bfsFS as default };
//src/bfsProcess.js
import { BFSRequire } from "browserfs";

const bfsProcess = BFSRequire("process");

export default bfsProcess;
AntonOfTheWoods commented 11 months ago

@meponderR - why close this? I wasted quite a few minutes finding this, and no one cares about what they have in the readme anymore... Wouldn't it be better to have this in the readme directly?

james-pre commented 11 months ago

@AntonOfTheWoods

Thanks for the interest. We have moved over to using esbuild for the build system so this issue is no longer relevant.

AntonOfTheWoods commented 11 months ago

@james-pre thanks for your quick response. I'm not sure I follow though. Are you saying that when you get the NPM creds, make_dist is fixed and produces a usable dist directory, and you have published to NPM, it will "just work" with the modern bundlers?

I spent several hours trying to get it to work and many of the adapters simply didn't work, probably because the bundler wasn't getting the right things in the right order. As an aside, nothing at all works when you use the vite-react plugin. If it all just works when a new release is made, fantastic!

james-pre commented 11 months ago

The make_dist command should not be used at all. You should only do npm run build.

AntonOfTheWoods commented 11 months ago

Thanks. I spent a while more and still couldn't get it to work. Trying to run using the var fs = BrowserFS.BFSRequire("fs"); var path = BrowserFS.BFSRequire("path"); versions only gets me as far as ReferenceError: Buffer is not defined. I spent a while throwing stuff at the wall but nothing stuck. Otherwise, I can't even get vite to compile anything. Understanding monkeypatching when you only have old docs is above my paygrade!

I will just have to wait patiently for the build and docs. Thanks.

james-pre commented 11 months ago

@AntonOfTheWoods Please try to build from my fork. I successfully build and can use methods. I've fixed some stuff but have not merged into the main repo yet.

AntonOfTheWoods commented 11 months ago

Thanks @james-pre . I was able to get it built and did a local install. Doing that didn't appear to make types available but did give me a working AsyncMirror using Indexeddb and InMemory! I tried hacking around (I have never understood node_modules and types/dependencies get me physically angry every time I have to deal with it... it seems to embody all of the nasty things you can find in the technology realm...) and managed to get types (by pointing to the src/index.ts and adding an exports).

However, I still don't understand how "native" fs & co. are supposed to work with vite. If you are no longer supposed to need any rollup plugins or the like, then that feature doesn't work with my setup. I really need an example to work from... :-).

There was also an issue with getting the FileSystemAccess to work - it fails when reading a file. I would try and debug but I simply don't understand the build system well enough to know what is a setup issue and what is code. When monkeypatching is involved and you don't understand how it is being set up... that is a recipe for broken cups on walls :-).

I'll have another look when you publish, and thanks again!

james-pre commented 11 months ago

You will want to use the returned values of BFSRequire as the alias in your build systemm You may need to use a plugin to replace fs with BrowserFS. I apologize for any confusion, I hadn't fully read the code in the initial issue comment.

AntonOfTheWoods commented 11 months ago

Thanks @james-pre , so presumably we will need shims like the make_dist used to produce?

james-pre commented 10 months ago

@AntonOfTheWoods

You should be able to just use the FSModule returned by BFSRequire without shims, albeit it does not have some of the latest Node FS functions.

AntonOfTheWoods commented 10 months ago

Thanks @james-pre . I, like many users I would imagine, would like to use browserfs to reuse existing libraries that actually use fs (and path/buffer I guess) for something meaningful, but where that "something meaningful" could also be on a virtual FS in a browser, but the original authors developed without that usecase in mind.

I, for example, would like to try and use sql-language-server (at heart a live sql completion and validation vscode language server written for node) in a Webworker, with WASM sqlite in another webworker. Now that webworkers have performant sync file access, sqlite in the browser is starting to make a huge amount of sense, and providing autocomplete and linting/validation completely in the browser (for my PhD project actually...) in a webworker is just too awesome to not give a try! I'm sure there will be lots more people wanting to use node fs stuff...

My understanding was that I would need shims for that. Or am I mistaken?

james-pre commented 10 months ago

@AntonOfTheWood

Please take a look at src/core/FS.ts. That is the compatibility layer, which makes sure browsers is a drop replacement for Node's FS (minus some of the newer functions I mentioned earlier). Before I became a maintainer of the project a few months ago, it had not received any commits in 3 years, and had a build system 6-7 years old. My biggest concern has been getting the tooling updated before I start to work on more of the codebase, that way I get formatting, linting, and CI/CD that work. I intend to get the project back to being 100% compatible with Node's FS.

As for usage within your own project, you can use a replacement plugin (I use esbuild for my projects which has the alias config option).

I hope you find success with this, but if it does not work and you think it is an issue with BFS, feel free to open a new issue.