Cannot parse `<T>(t: T) => T` under JSX mode (problems with expression prefix)

[kaleidawave]

The JSX lexing kicks in after the previous token is considered a expression prefix (this works for JSX lexing). So for the colon :, JSX lexing mode kicks in

let x = { a: <*JSX mode* ... }
let y = ... ? ... : <*JSX mode

IMO the former should be (:)= and the latter should be (if) ... then ... else ... but

This means that the following doesn't work:

function func(a: <T>(t: T) => T) {}
//             ^ ^^^
//               Thinks this is JSX because of the : before 

This also doesn't parse correctly

type X = (<T>(t: T) => T) | number;
//     ^

Not sure how to tackle this. The lexing is designed to be context-free. Communicating with the parser would require architectural changes. Maybe the lexer could hold a state so that it knows it is in some mode? Maybe could lookahead in the lexer and check that the next == '('?

It is a shame because the expression prefix mode works really well ATM and covers regular expression stuff.

I think this works on non JSX parsing mode?

Alternatively this works ATM

function func(a: { b<T>(t: T): T }["b"]) {
  print_type(a)
}

---

This is not to do with generic arrow function syntax (don't do this)

const x = <T>(t: T) => t;

or the casting syntax (never do this)

const x = <string>b;

[jasikpark]

Curious how esbuild parses it, since they obv support distinguishing between TSX types and elements declarations: https://github.com/evanw/esbuild/blob/fc37c2fa9de2ad77476a6d4a8f1516196b90187e/internal/js_parser/ts_parser_test.go#L556

Would be sweet to replicate these tests if possible too.

Replicating the tests here is probably fine since it's MIT? https://github.com/evanw/esbuild/blob/main/LICENSE.md

[kaleidawave]

Hmm, may be a design problem. See the last two sentences.

https://github.com/evanw/esbuild/blob/fc37c2fa9de2ad77476a6d4a8f1516196b90187e/internal/js_lexer/js_lexer.go#L3-L8

// The lexer converts a source file to a stream of tokens. Unlike many
// compilers, esbuild does not run the lexer to completion before the parser is
// started. Instead, the lexer is called repeatedly by the parser as the parser
// parses the file. This is because many tokens are context-sensitive and need
// high-level information from the parser. Examples are regular expression
// literals and JSX elements.

I could change it so that lexing "pauses" on RegExp and JSX. Then a channel from the parser could communicate where it is? Don't like the idea of that big of change considering it is only for an edge case.

---

That TS parser tests look good. Shouldn't be too hard to grep out and make some tests for under parser/tests

[kaleidawave]

A few more problems with the current "expression prefix" token thing for changing the lexing mode:

• for (const ... of /x/) works but that breaks x.of.y because it thinks .y is a number literal without the 0 prefix. (also others)
• and .return is also a valid property access so .return < 5 breaks in JSX mode

I think more 'state' in the lexer could solve this. There is current low level state, but a higher level state that captures the state here

• property access for (?).return
• for (await) (... of)
• others?

Might attempt in #173, might not? 🤷