Webpack Issues

[thekevshow]

Steps to Reproduce

1. Set up a Node.js project with @dsnp/parquetjs as a dependency.
2. Configure Webpack for the project with the following settings:
   • Target: 'node'
   • Output library target: 'umd'
   • Entry: [Point to the main file of the test package]
3. Include the import statement const { ParquetWriter } = require('@dsnp/parquetjs'); in the main file.
4. Run Webpack to bundle the project.
5. Execute the bundled code.

Expected Behaviour

The application should bundle without errors and the @dsnp/parquetjs module should be correctly imported and functional when running the bundled code.

Actual Behaviour

Upon running the Webpack bundle, the application throws an error: Error: Cannot find module 'util'. This suggests that Webpack is unable to resolve the util module, a core Node.js module, which is required by @dsnp/parquetjs or its dependencies.

Any Logs, Error Output, Etc.?

Error: Cannot find module 'util'
    at t (index.js:2:2329285)
    at 86275 (index.js:2:2327909)
    ... [Additional stack trace] ...
    at 19785 (index.js:2:565811) {
  code: 'MODULE_NOT_FOUND'
}

Any Other Comments?

• Attempts to resolve the issue by adding a fallback configuration for util in Webpack did not succeed.
• The project works as expected without Webpack bundling.
• This issue seems to arise specifically when using @dsnp/parquetjs in a Webpack-bundled Node.js environment.
• Any insights or recommendations on configuration changes or workarounds would be greatly appreciated.

[wilwade]

There might be an easier way, but I think you have to map it to the browser distribution via alias in the webpack config. Assuming you want to webpack it without also doing the browserification part as well.

via: https://github.com/LibertyDSNP/example-client/blob/main/config/webpack.config.js#L308-L316

webpack.config.js

return {
// ...
    alias: {
        "@dsnp/parquetjs": path.resolve(
            __dirname,
            "..",
            "node_modules",
            "@dsnp",
            "parquetjs",
            "dist",
            "browser",
            "parquet.cjs.js"
        ),
    // ...
    }

[thekevshow]

This still doesn't seem to work causing an error

TypeError: Cannot read properties of undefined (reading 'webkitRequestFileSystem')

@wilwade

[wilwade]

Hmm..

1. Can you share your webpack config file?
2. What version of webpack are you using? My example was from Webpack 4. I'm not sure what the equivalent would be in Webpack 5.

[thekevshow]

Here is what the webpack config looks like

const path = require('path');
const libExternals = {
}
const streamWorkerPackage = {
    entry: ['./packages/stream-workers/index.js'],
    target: 'node',
    output: {
        path: `${process.cwd()}/bin/stream-workers`,
        filename: 'index.js',
        libraryTarget: 'umd'
    },
    resolve: {
        symlinks: false,
        extensions: ['.js', '.json'],
        alias: {
            "@dsnp/parquetjs": path.resolve(__dirname, 'node_modules', '@dsnp/parquetjs', 'dist', 'browser', 'parquet.cjs.js')

            // "@dsnp/parquetjs": path.resolve(
            //     __dirname,
            //     "..",
            //     "node_modules",
            //     "@dsnp",
            //     "parquetjs",
            //     "dist",
            //     "browser",
            //     "parquet.cjs.js"
            // ),
        }
    },
    externals: {
        ...libExternals
    }
}
module.exports = [
    streamWorkerPackage
];

and I am using versions listed here:

"webpack": "5.90.1",
"webpack-cli": "5.1.4"

[wilwade]

@thekevshow Hmm... Is this for a library or for a direct web application?

If it is a library, then I would suggest a different route to get a better build: Handle the shims directly. So Parquetjs takes a lot of node libraries to work. The browser edition is designed to be used directly in a script tag, or in some cases in a build where you know there are no need for the build to otherwise shim those node packages.

Might also help to get a trace for that TypeError: Cannot read properties... error. This sounds like a Webkit issue and I am very out of date on my webkit.

[thekevshow]

This is not for a web application it is for an AWS lambda which we are webpacking to bundle everything into for cost saving and cold time reducing pieces here. For all intensive purposes it is a nodejs application. Testing was done locally with the webpack result. @wilwade but that makes sense appreciate it.