search

LibertyDSNP / parquetjs

Fully asynchronous, pure JavaScript implementation of the Parquet file format with additional features
MIT License
43 stars 24 forks source link

webkitRequestFileSystem and brotli_wasm_bg.wasm problem #125

Open szanuje opened 2 months ago

szanuje commented 2 months ago

Thanks for reporting an issue!

Steps to reproduce

Add package to NextJS 14 project

Expected behaviour

Package imported correctly

Actual behaviour

Package is throwing error on NextJS build

Any logs, error output, etc?

Error is either:

Error: ENOENT: no such file or directory, open '/Users/<projectPath>/.next/server/chunks/brotli_wasm_bg.wasm'

or

 Collecting page data  ...TypeError: Cannot read properties of undefined (reading 'webkitRequestFileSystem')

depending on which import statement I use:

import parquetjs from "@dsnp/parquetjs";
import parquetjs from "@dsnp/parquetjs/dist/browser/parquet.esm";

Any other comments?

...

wilwade commented 2 months ago

@szanuje Which version of webpack are you using? We've had a few issues with correctly handling webpack that might be related:

JasonYeMSFT commented 2 months ago

'brotli-wasm' requires the wasm binary in a way that is very unfriendly to web bundlers.

const path = require('path').join(__dirname, 'brotli_wasm_bg.wasm');
const bytes = require('fs').readFileSync(path);

The dynamic fs read prevents the bundler from bundling the wasm, although we implemented a custom wasm loader. The root cause seems to be this https://github.com/rustwasm/wasm-pack/issues/831