search

guybedford / es-module-lexer

Low-overhead lexer dedicated to ES module parsing for fast analysis
MIT License
912 stars 47 forks source link

Raw C usage (or in Go via Cgo) #147

Open b3nten opened 1 year ago

b3nten commented 1 year ago

Hi,

I don't have experience with wasm (or C for that matter) but I'm wondering how difficult this would be to get working in C (and eventually Go)? Would it be a few small changes, or is it entirely designed for wasm as a target and would require significant work?

End goal would be to parse JS strings with this in a Go cli.

Cheers

guybedford commented 1 year ago

It should be possible to get this project building for native C quite easily yes. That said, since it wasn't designed for this, it likely would have vulnerabilities outside of being used in a Wasm sandbox.

There is also a Rust build at https://github.com/jspm/rust-lexer which may be easier to run in Go.

Another alternative might even be to use the component build at https://github.com/guybedford/es-module-lexer/pull/143 which builds a WebAssembly component which could be used in Go while retaining strong security. These Wasm directions are very experimental though so it would only be if you're interested in exploring that.

b3nten commented 1 year ago

Hi, thanks for the response

The Rust build looks promising, although I can't find it (perhaps it's a private repo?). I thought about using WASM but that looks like it would be fairly complex to run. I'm willing to explore different routes - it's not a high priority, I have the functionality via EsBuild but this would be preferable. What's the experimental part with Wasm?

guybedford commented 1 year ago

Ah, ok I've given you access to the Rust lexer. Note it's behind this main branch by quite a bit now, although in theory the upstreaming shouldn't be too tricky. Open to collaboration on that repo. Also happy to share more about Wasm workflows too.

thepassle commented 8 months ago

I'd be really interested in the Rust lexer as well (or other documentation on how to use the wasm module in rust). I was trying to get the lexer to run in Rust, but I was having a hard time using both wasmtime and cc, and can't seem to get things up and running.

thepassle commented 8 months ago

Also found this today: https://github.com/devongovett/es-module-lexer/blob/main/src/lib.rs for anybody else who's reading along

guybedford commented 8 months ago

The Rust version worked on previously in collaboration with @vovacodes is at https://github.com/jspm/rust-lexer. It hasn't been updated in over two years to the current es-module-lexer commits though.