kaleidawave / ezno

A JavaScript compiler and TypeScript checker written in Rust with a focus on static analysis and runtime performance
https://kaleidawave.github.io/posts/introducing-ezno/
MIT License
2.3k stars 42 forks source link

Cannot parse `<T>(t: T) => T` under JSX mode #165

Open kaleidawave opened 2 weeks ago

kaleidawave commented 2 weeks ago

The JSX lexing kicks in after the previous token is considered a expression prefix (this works for JSX lexing). So for the colon :, JSX lexing mode kicks in

let x = { a: <*JSX mode* ... }
let y = ... ? ... : <*JSX mode

IMO the former should be (:)= and the latter should be (if) ... then ... else ... but

This means that the following doesn't work:

function func(a: <T>(t: T) => T) {}
//             ^ ^^^
//               Thinks this is JSX because of the : before 

This also doesn't parse correctly

type X = (<T>(t: T) => T) | number;
//     ^

Not sure how to tackle this. The lexing is designed to be context-free. Communicating with the parser would require architectural changes. Maybe the lexer could hold a state so that it knows it is in some mode? Maybe could lookahead in the lexer and check that the next == '('?

It is a shame because the expression prefix mode works really well ATM and covers regular expression stuff.

I think this works on non JSX parsing mode?

Alternatively this works ATM

function func(a: { b<T>(t: T): T }["b"]) {
  print_type(a)
}

This is not to do with generic arrow function syntax (don't do this)

const x = <T>(t: T) => t;

or the casting syntax (never do this)

const x = <string>b;
jasikpark commented 2 weeks ago

Curious how esbuild parses it, since they obv support distinguishing between TSX types and elements declarations: https://github.com/evanw/esbuild/blob/fc37c2fa9de2ad77476a6d4a8f1516196b90187e/internal/js_parser/ts_parser_test.go#L556

Would be sweet to replicate these tests if possible too.

Replicating the tests here is probably fine since it's MIT? https://github.com/evanw/esbuild/blob/main/LICENSE.md

kaleidawave commented 1 week ago

Hmm, may be a design problem. See the last two sentences.

https://github.com/evanw/esbuild/blob/fc37c2fa9de2ad77476a6d4a8f1516196b90187e/internal/js_lexer/js_lexer.go#L3-L8

// The lexer converts a source file to a stream of tokens. Unlike many
// compilers, esbuild does not run the lexer to completion before the parser is
// started. Instead, the lexer is called repeatedly by the parser as the parser
// parses the file. This is because many tokens are context-sensitive and need
// high-level information from the parser. Examples are regular expression
// literals and JSX elements.

I could change it so that lexing "pauses" on RegExp and JSX. Then a channel from the parser could communicate where it is? Don't like the idea of that big of change considering it is only for an edge case.


That TS parser tests look good. Shouldn't be too hard to grep out and make some tests for under parser/tests