antlr / antlr4

ANTLR (ANother Tool for Language Recognition) is a powerful parser generator for reading, processing, executing, or translating structured text or binary files.
http://antlr.org
BSD 3-Clause "New" or "Revised" License
17.03k stars 3.27k forks source link

Esbuild fail to bundle from 4.10 to 4.12 #4211

Open richard-julien opened 1 year ago

richard-julien commented 1 year ago

Please include the following information:

Hi Antlr team,

I will try to expose my problem in a clearer way but sorry if information looks like partial. Difficult to explain correctly my problem with my current knowledge of ANTLR.

I try to upgrade ANTLR in my project (nodejs) from 4.10 to 4.12. My project use esbuild to bundle the source code. After upgrading to 4.12, the bundled code refuse to start with the error.

node:internal/modules/cjs/loader:1367
    throw new ERR_INVALID_ARG_VALUE('filename', filename, createRequireError);
    ^

TypeError [ERR_INVALID_ARG_VALUE]: The argument 'filename' must be a file URL object, file URL string, or absolute path string. Received undefined

Looking into the bundled code, its looks like the esbuild transform the file and i have this line.

var ha = (0, import_module.createRequire)(import_meta.url)("fs");

And as the error said, the import_meta is empty initialized so url is undefined.

Looking into esbuild, the documentation said that import.meta.url is not supported when bundling:

You also may not want to bundle your dependencies with esbuild. There are many node-specific features that esbuild doesn't support while bundling such as __dirname, import.meta.url, fs.readFileSync, and *.node native binary modules. You can exclude all of your dependencies from the bundle by setting [packages](https://esbuild.github.io/api/#packages) to external:

So my question is, do you change something in ANTLR that now use the meta import feature that break the compatibility with esbuild bundling?

Thanks in advance.

ericvergnaud commented 1 year ago

Yes the packaging changed a bit in 4.12 for TypeScript to work.

richard-julien commented 1 year ago

Any solution/workaround available @ericvergnaud? Problem is still present in 4.13 and i dont know where to start to upgrade.

Tarjei400 commented 1 year ago

@richard-julien How you build your app? If you could provide some small repo with your setup that reproduces this error I can assist you with that.

ericvergnaud commented 1 year ago

This might already be fixed, a release is planned this week-end, let's see if it happens and does fix the problem.

Tarjei400 commented 1 year ago

@ericvergnaud Cool! Just in case let me know guys if there are issues, would gladly help :+1:

ShaharCider commented 1 year ago

@ericvergnaud Hey, has this been released yet?

ZenMagus commented 1 year ago

i'm seeing the same issue using esbuild - so am also curious about release status/timeline

ZenMagus commented 1 year ago

npm install antlr4 --save-dev npm install esbuild --save-dev

--antlr4@4.13.1 --esbuild@0.19.3

npx esbuild target.js --bundle --platform=node --format=cjs --outfile=out.cjs

---> node out.cjs inFile

<--- TypeError [ERR-INVALID_ARG_VALUE]: The argument 'fliename' must be a file URL object, file URL string, or absolute path string. Received undefined

ericvergnaud commented 1 year ago

Can you try manually replacing antlr4's package.json with latest from 'development' branch and see of it fixes the problem ?

ZenMagus commented 1 year ago

Hey! thanks for the idea to try! ---> OK here is what i found:

looks like the package.json installed via [ npm install antlr4 ] is from master and has the following:

"browser": "dist/antlr4.web.js", "main": "dist/antlr4.node.mjs",

and the package.json from dev has the following changes:

c "browser": "dist/antlr4.web.mjs", c "main": "dist/antlr4.node.cjs",

-- when i place the updated 'dev' package.json in /node_modules/antlr4 and re-run esbuild it still results in the undefined filename error

-- i am willing and able to work on this -- do you have ideas for further adjustments to experiment with?

maybe i can install the entire dev branch? but not exactly sure how to do that without NPM -- i'm search up how to do this and see if that will fix it.

ericvergnaud commented 1 year ago

The dev file seems to work for many people, I have never used esbuild so I encourage you to try out things Maybe what's missing is a esbuild cli option ? also not sure what target.js is (filename error raises all sorts of suspicions)

ZenMagus commented 1 year ago

is my first time trying to pack javascript -- tried and failed with rollup and esbuild so far. target.js name was just to obscure actual file name. i believe the error i'm experiencing is the same error that is described in detail at the beginning of this thread by the OP. basically i have a working javascript ANTLR4 parser - but it breaks when i try to pack it up. want to move it into a VSCode extension without requiring ANTLR4 being required by the extension - but may have to go that route if i can't pack it up.

ericvergnaud commented 1 year ago

The generated parser requires the antlr runtime, regardless of packaging

ZenMagus commented 1 year ago

ah OK - i thought i could pack it all up -- is probably better form just to make antlr4 npm install part of the VSCode dependency/environment so will go that route -- thank you for your thoughts and your help! very much appreciated <3

eirueirufu commented 11 months ago

@ZenMagus i met the same problem, it seems to be caused by the import.meta.url, You can see if this solution can help you https://github.com/evanw/esbuild/issues/1492#issuecomment-893144483

ericvergnaud commented 10 months ago

All, please try 4.13.1-patch-1, so we can possibly close this.

ZenMagus commented 10 months ago

Hi @ericvergnaud @eirueirufu

Thank you for your responses and input!

My dev environment is no longer in context with the OPs issue so i don't have follow up.

Since Ericvergnaud's reply above I have gone the route of

ANTLR4 is amazing.
I am grateful to be able to have this tool - thank you.

Janther commented 9 months ago

The bug is still in this repo but I found a way to inject a fix at the command level in this comment :

https://github.com/evanw/esbuild/issues/2441#issuecomment-1315341698