Support TypeScript

[Thom1729]

Like it says. I'm not sure how much of an effort this would be.

Known bugs/missing features

• Old-style casting (<Foo> bar). This conflicts with JSX, so I'll have to make sure that this is configurable. Implemented in beta.3
• Type arguments in method calls. On the surface, this is ambiguous with ordinary JS syntax. I'll have to look into exactly how TypeScript parses this. May require branching.
• Type arguments in template strings. Same concerns as above. May also require a special case for Styled Components.
• Better detection of arrow function argument lists. May require branching.
• Arrow function type parameters. Almost certainly require branching.
• Directives in comments. For this, I'll need to figure out a general system for extending comments that works for other extensions as well.

[michaelblyons]

Low impact: you're missing <MyType>foo-style casting (equivalent to foo as MyType), and some generics like

• foo = new Foo<MyType>()
• bar = Bar<MyType>()
• class Foo extends Bar<MyType> {
• class Foo implements Bar<MyType> {

These only have local highlighting problems and do not appear to break anything downstream.

[Thom1729]

The latter two should be easy to fix. I haven't implemented function call type arguments yet; the syntax looks ambiguous and I'm not sure exactly how TypeScript differentiates it. I have a hunch it may require branching.

On Tue, Jun 23, 2020, 12:01 Michael notifications@github.com wrote:

Low impact: you're missing foo-style casting (equivalent to foo as MyType), and some generics like

• foo = new Foo()
• bar = Bar()
• class Foo extends Bar {
• class Foo implements Bar {

These only have local highlighting problems and do not appear to break anything downstream.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Thom1729/Sublime-JS-Custom/issues/51#issuecomment-648258303, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEBMII5QH6IPM2QVBQPTLVDRYDGWDANCNFSM4GHWHLAA .

[tsujp]

I believe that latest would also apply to the styled-components scope too then @michaelblyons? seen here: https://github.com/Thom1729/Sublime-JS-Custom/issues/93#issuecomment-646946192

[michaelblyons]

@tsujp 🤷‍♂️ I do not know. I do not have code that looks like that.

[Thom1729]

v2.4.0-beta.3 is out supporting generic arguments in extends and implements clauses.

There's a new typescript.old_style_assertions option to enable <T>foo assertions. The interface there is not final, but it should work for now. I suppose that I could have the TypeScript extension check whether the JSX extension is enabled, but extensions weren't designed to interact in that way and it would be rather a kludge.

[Thom1729]

I'm working on the documentation. (I won't merge it until the end, since the front page is from master.) Here's the first draft of the TypeScript docs:

typescript: boolean (Beta)

Highlight TypeScript. TypeScript support is currently in beta, meaning that there may be bugs and that the exact behavior is subject to change.

By default, old-style type assertions (e.g. <T>foo) are not highlighted. You can enable them via the typescript.old_style_assertions option. As an example, these user preferences will provide configurations for both plain TypeScript and TypeScript with JSX:

{
    "configurations": {
        "TypeScript": {
            "file_extensions": [ "ts" ],
            "typescript": {
                "old_style_assertions": true
            },
        },
        "TypeScript (JSX)": {
            "file_extensions": [ "tsx" ],
            "typescript": true,
            "jsx": true
        }
    }
}

[Thom1729]

Microsoft's TypeScript syntax definition uses the scope source.ts instead of source.js. Is this a useful feature for e.g. tool compatibility?

[rchl]

Yes. For language servers we have to be able to differentiate between typescript and javascript files and that is done now using the main scope of the syntax.

[michaelblyons]

Microsoft's TypeScript syntax definition uses the scope source.ts instead of source.js. Is this a useful feature for e.g. tool compatibility?

I'd say "yes." I made a PR on FileIcons to mark source.js.typescript as TypeScript, but mimicking the Microsoft base scope will obviate any need to do that for other packages that expect source.ts. This is particularly important since what I added depends on the JSC variant being named "TypeScript."

[predragnikolic]

You can configure the base scope with JSCustom with the scope config option.

this will change the base scope from source.js.typescript to source.ts

"TypeScript": {
            "file_extensions": [ "ts" ],
            "typescript": true,
            "scope": "source.ts",
        },
        "TypeScriptReact": {
            "file_extensions": [ "tsx" ],
            "typescript": true,
            "scope": "source.tsx",
            "jsx": true
        }

[Thom1729]

@rchl @michaelblyons Is changing the root scope sufficient or is it important that all of the individual scopes also end with .ts?

[michaelblyons]

Heh. For the things I've run into so far, @predragnikolic's nudge toward the README that I didn't read is entirely sufficient. 😁

[rchl]

Is changing the root scope sufficient or is it important that all of the individual scopes also end with .ts?

Good question. For language servers at least, I believe we never really had to target individual scopes specifically so the suffix of those shouldn't matter. Maybe @rwols can confirm.

I've never even noticed that it's like that for both TS and default JS syntax...

For file icons, I believe only base scope matters.

[Thom1729]

PR #97 would allow replacing .js with .ts or .tsx anywhere in the syntax. If it does turn out to be needed, I'll have to add some tests/do cleanup.

[rwols]

For language servers at least, I believe we never really had to target individual scopes specifically so the suffix of those shouldn't matter.

I can confirm only the base scope matters.

It doesn't matter for LSP what the base scope will be. You can set source.tsx | source.ts.react as document selector to get it to work with both Microsoft's tmLanguage file as well as Thom's syntaxes (at least, for ST4. For ST3 you have to fill in the plain old path to the syntax file).

[Thom1729]

v2.4.0-beta.4 is out with the following improvements:

• Add default configurations for TypeScript, with and without JSX.
• Fix a bug where JS Custom would try to rebuild a deleted configuration, causing an error.
• When rebuild_js_custom_syntaxes is called with a versions argument, don't delete syntaxes that don't match versions.
• Add separate tests for TypeScript with and without JSX.

[Thom1729]

Fun fact: in order to support function call type arguments, Microsoft's official TypeScript syntax uses the following regular expressions:

(?=(((([_$[:alpha:]][_$[:alnum:]]*)(\s*\??\.\s*(\#?[_$[:alpha:]][_$[:alnum:]]*))*)|(\??\.\s*\#?[_$[:alpha:]][_$[:alnum:]]*))|(?<=[\)]))\s*(?:(\?\.\s*)|(\!))?((<\s*(((keyof|infer|awaited|typeof|readonly)\s+)|(([_$[:alpha:]][_$[:alnum:]]*|(\{([^\{\}]|(\{[^\{\}]*\}))*\})|(\(([^\(\)]|(\([^\(\)]*\)))*\))|(\[([^\[\]]|(\[[^\[\]]*\]))*\])|(\'([^\'\\]|\\.)*\')|(\"([^\"\\]|\\.)*\")|(\`([^\`\\]|\\.)*\`))(?=\s*([\<\>\,\.\[]|=>|&(?!&)|\|(?!\|)))))([^<>\(]|(\(([^\(\)]|(\([^\(\)]*\)))*\))|(?<==)\>|\<\s*(((keyof|infer|awaited|typeof|readonly)\s+)|(([_$[:alpha:]][_$[:alnum:]]*|(\{([^\{\}]|(\{[^\{\}]*\}))*\})|(\(([^\(\)]|(\([^\(\)]*\)))*\))|(\[([^\[\]]|(\[[^\[\]]*\]))*\])|(\'([^\'\\]|\\.)*\')|(\"([^\"\\]|\\.)*\")|(\`([^\`\\]|\\.)*\`))(?=\s*([\<\>\,\.\[]|=>|&(?!&)|\|(?!\|)))))(([^<>\(]|(\(([^\(\)]|(\([^\(\)]*\)))*\))|(?<==)\>|\<\s*(((keyof|infer|awaited|typeof|readonly)\s+)|(([_$[:alpha:]][_$[:alnum:]]*|(\{([^\{\}]|(\{[^\{\}]*\}))*\})|(\(([^\(\)]|(\([^\(\)]*\)))*\))|(\[([^\[\]]|(\[[^\[\]]*\]))*\])|(\'([^\'\\]|\\.)*\')|(\"([^\"\\]|\\.)*\")|(\`([^\`\\]|\\.)*\`))(?=\s*([\<\>\,\.\[]|=>|&(?!&)|\|(?!\|)))))([^<>\(]|(\(([^\(\)]|(\([^\(\)]*\)))*\))|(?<==)\>)*(?<!=)\>))*(?<!=)\>)*(?<!=)>\s*)?\())

(?<=\))(?!(((([_$[:alpha:]][_$[:alnum:]]*)(\s*\??\.\s*(\#?[_$[:alpha:]][_$[:alnum:]]*))*)|(\??\.\s*\#?[_$[:alpha:]][_$[:alnum:]]*))|(?<=[\)]))\s*(?:(\?\.\s*)|(\!))?((<\s*(((keyof|infer|awaited|typeof|readonly)\s+)|(([_$[:alpha:]][_$[:alnum:]]*|(\{([^\{\}]|(\{[^\{\}]*\}))*\})|(\(([^\(\)]|(\([^\(\)]*\)))*\))|(\[([^\[\]]|(\[[^\[\]]*\]))*\])|(\'([^\'\\]|\\.)*\')|(\"([^\"\\]|\\.)*\")|(\`([^\`\\]|\\.)*\`))(?=\s*([\<\>\,\.\[]|=>|&(?!&)|\|(?!\|)))))([^<>\(]|(\(([^\(\)]|(\([^\(\)]*\)))*\))|(?<==)\>|\<\s*(((keyof|infer|awaited|typeof|readonly)\s+)|(([_$[:alpha:]][_$[:alnum:]]*|(\{([^\{\}]|(\{[^\{\}]*\}))*\})|(\(([^\(\)]|(\([^\(\)]*\)))*\))|(\[([^\[\]]|(\[[^\[\]]*\]))*\])|(\'([^\'\\]|\\.)*\')|(\"([^\"\\]|\\.)*\")|(\`([^\`\\]|\\.)*\`))(?=\s*([\<\>\,\.\[]|=>|&(?!&)|\|(?!\|)))))(([^<>\(]|(\(([^\(\)]|(\([^\(\)]*\)))*\))|(?<==)\>|\<\s*(((keyof|infer|awaited|typeof|readonly)\s+)|(([_$[:alpha:]][_$[:alnum:]]*|(\{([^\{\}]|(\{[^\{\}]*\}))*\})|(\(([^\(\)]|(\([^\(\)]*\)))*\))|(\[([^\[\]]|(\[[^\[\]]*\]))*\])|(\'([^\'\\]|\\.)*\')|(\"([^\"\\]|\\.)*\")|(\`([^\`\\]|\\.)*\`))(?=\s*([\<\>\,\.\[]|=>|&(?!&)|\|(?!\|)))))([^<>\(]|(\(([^\(\)]|(\([^\(\)]*\)))*\))|(?<==)\>)*(?<!=)\>))*(?<!=)\>)*(?<!=)>\s*)?\())

I was going to borrow Microsoft's lookaheads as a temporary measure before reimplementing with branching, but upon reflection, I think not.

[predragnikolic]

I wonder if the regular expression was written by a human?

[wbond]

I’m glad we’ve got a more maintainable solution moving forward.

[Thom1729]

Yeah, it cannot be exaggerated how big a deal branching is for TypeScript parsing.

[tsujp]

Holy code batman.

[Thom1729]

v2.4.0 is out. I still need to fix #98 and some related issues, but once those are resolved I'll be opening a 3.0 branch that uses branching to implement the remaining TypeScript features.

[Thom1729]

There is now a sublime-4 branch based on the most recent core syntax. In addition, type arguments are supported in function calls and template tags. (At the moment, the function name/template tag is marked as a plain variable rather than function.)

[tsujp]

@Thom1729 I've noticed a drop in syntax highlighting for multi-line object inline-types when paired with function components in React.

Example code with syntax breaking after the first line:

export const FooComponent = ({
  some,  // syntax highlighting which was present before now breaks at this line and onwards
  list,
  of,
  props,
}: {
  some: string
  list: string
  of: boolean
  props: number
}) => {
  // not important
}

[Thom1729]

I've just published v3.0.0-alpha.2. The 3.0.0 branch requires a prerelease Sublime version, but it should provide much better arrow function detection.

[tsujp]

Fixed on those.

[Thom1729]

For context, without the new branching feature in the prereleases, Sublime's parser can't look ahead over line boundaries. This means that when it sees export const FooComponent = ({, it has to guess whether the opening paren is part of a parenthesized expression or an arrow function. When in doubt, it would always guess a parenthesized expression, and there is special logic later on so that when it saw the =>, it would realize its mistake and highlight the remainder as an arrow function. Presumably, there was some flaw with this logic and the colon confused it.

The prereleases have a new branching feature. When the parser sees the =>, it can backtrack and highlight the entire arrow function properly. This should allow us to identify arrow functions with 100% accuracy — any bugs that remain should be properly fixable, rather than resorting to arcane lookaheads and fallbacks.

[tsujp]

Very interesting, and nice work too to both you and the ST4 devs.

[Thom1729]

I'm closing this because the beta release should fix everything in the initial psot.