TypeScript compiler integration.

[nidin]

In order to move fast forward we need to integrate turboscript with typescript compiler. I will create a new branch for it.

[MichaReiser]

Hmm... that's the one decision I regret. It made the code horribly complicated, and the compiler supports the wrong features.

Main issues I had

• The API is just not intended to be used for a compiler. It is mainly a type checker. You will face many situations where the API just does not provide the information you need, or you need to hack around to get to the information
• There is no API documentation
• TypeScript lacks support for many base types like int, uint...
• TypeScript is unsound
• TypeScript supports features that cannot be compiled efficiently to native code --- as does JavaScript. The result thereof is that TurboScript will be slower than JavaScript. Furthermore, you still need your own analyzer to determine any unsupported features. Here I strongly encourage you starting benchmarking and comparing your results with JS soon to ensure that performance does not drop because some unessential feature has been added.

I think, having your own simple type checker is actually a benefit.

[nidin]

Good points. These were the initial reasons I don't choose modifying TypeScript compiler. Mainly I don't understand TypeScript compiler due to lack of documentation. 🙏 Thanks for the valuable input.

[MichaReiser]

The TypeScript API is quite simple in most cases. However, it's often unclear what result you get (in some cases you will find out that the result seems not to return what you have expected but behaves well for others).

However, I understand that you don't want to maintain your own type checker. But at the moment I don't know of any TypeChecker that has support for "low-level types". Supporting these low-level types has the disadvantages that TurboScript (or Speedy.js) does not embed nicely into TypeScript or Flow because it supports additional types. Right know, I don't know what the best solution is. My goal would be to have my own type checker that can parse either TypeScript or Flow code but also allows to have base types like int... However, it does not seem as TypeScript intends to add support for more base types in the near future and therefore, the IDEs will show a bunch of errors if unsupported base types are used. So, I begin to believe that separating the JavaScript and the TurboScript code might have been the better chose than allowing directly embedding it into TypeScript code. Even though I believe the latter is far more elegant.

So, I'm very interested in where this is going as I'm facing the same issues.

[nidin]

For the time being I will continue supporting current TurboScript type checker. The primary goal is not supporting complete TypeScript instead make TypeScript like language (TurboScript) faster in WebAssembly

[izengliang]

I think language grammar and typescript the same, while adding new types, and for efficiency do not have to support all the typescript syntax, if all support is better.

"use turboscript";
class User{
......
}

[MaxGraey]

@MichaReiser in speedy.js you are using custom transform for visitor of AST Walker, right? But what if using someting lower level like this: https://github.com/Microsoft/TypeScript/blob/02547fe664a1b5d1f07ea459f054c34e356d3746/tests/cases/conformance/parser/ecmascript5/parserRealSource11.ts

[MichaReiser]

@MaxGraey The AST walking part isn't the problem. Most compiler use AST walking for generating the source code. The custom transform is needed to remove the source of the compiled function from the emitted code and to rewrite the entry functions (a function that is called from pure js).

The main problem of the TypeScript API are the missing base types. These are one of the main reasons why WebAssembly achieves better performance compared to a pure JS implementation. Today's JavaScript engines are surprisingly fast! Without good consideration of the language features and heavy offline optimization, the WebAssembly implementation is most certainly slower than a pure JS implementation (when talking about peak performance).

[winksaville]

What about using LLVM?

[MichaReiser]

@winksaville I believe that is a totally different topic, isn't it?

I believe LLVM is great and offers many optimizations for free. However, it requires that the users have an llvm installation, making the setup much harder --- especially as the web assembly backend is not yet part of the prebuilt binaries. I personally would use the optimizer that is part of binaryen and compare the runtime of TurboScript implementations to ones in Emscripten. If there is a significant difference, you may consider switching to LLVM.

[MaxGraey]

@MichaReiser so implement for example SyntaxKind.Float32Keyword, and create it as intrinsic type is huge problem?

[MaxGraey]

@winksaville someone told that emscripten or other tool allow optimize wasm files. So we can generate unoptimized wasm and optimize it later if needed.

[MichaReiser]

@MaxGraey Yes it is. You need to maintain your own version of TypeScript. And, since TypeScript is written in TypeScript you may need to make further adjustments to the TypeScript code just that it still compiles. I don't know if this is any better than maintain a custom, very simple type checker instead?

However, I would say create a simple prototype. See if you face the same issues as I did and do some benchmarking. Maybe, you find some trick I did not or you can gain the favor of the TypeScript team so that they add support for base types.

[winksaville]

@MichaReiser it is a different subject, but it seemed that using TypeScript to generate wasm wasn't practical based on your comments so I just threw it out. And you're right there are no prebuilt binaries yet but apparently it will be in the next release. I have compiled the a tool chain using these instructions and it was able to compile a couple trivial things. So its coming along.

@MaxGraey my assumption is that in the foreseeable future LLVM will probably have the best tool set.

[MichaReiser]

@winksaville That's the toolchain I'm using in Speedy.js

[nidin]

I would wait for TypeScript team's official support for low level types, otherwise we might need to spent more time on patching TypeScript. At this point I would suggest freeze adding features to TurboScript and make current version bug free. @MichaReiser I dropped class inheritance for now. I need to fix the issue with generics and we will make some benchmarks against TypeScript version then we can move forward. These initial benchmarks will be useful for further developments.

On Wed 17. May 2017 at 5:46 PM, Micha Reiser notifications@github.com wrote:

@winksaville https://github.com/winksaville That's the toolchain I'm using in Speedy.js (see draft of documentation https://github.com/MichaReiser/speedy.js-paper/releases/latest)

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/01alchemist/TurboScript/issues/87#issuecomment-302132960, or mute the thread https://github.com/notifications/unsubscribe-auth/AAMGZjTnwF39puhuk3Chs-mMN-KWXMSOks5r6xZWgaJpZM4Nb-ZN .

[winksaville]

@MichaReiser an FYI its confusing to me that you say Speedy.js is a JavaScript to wasm compiler but then you say it takes TypeScript source.

So I think @nidin's goal "TypeScript like language (TurboScript) faster in WebAssembly" is the good goal.

[izengliang]

I find it https://github.com/jeremyfa/node-ts2hx
have help ?

Typscript -> c++ -> wasm ?

[izengliang]

find it https://github.com/andrei-markeev/ts2c

typescript -> c -> wasm ?

[MaxGraey]

@liangzeng I don't think it's a good idea. A lot of tools and compile time and we should support 3-4 tools in the same time instead just one

[MaxGraey]

By the way this tools stack in my opinion js -> haxe -> c -> emscripten -> wasm already used in NectarJS and it has a huge bundle size

[izengliang]

ts -> wast -> wasm
Whether to reduce the workload?

[MaxGraey]

ts2c is completely unusable. For example it can't transpile this simple example:

class Vec3 {
   x: number;
   y: number;
   z: number;

   constructor(x: number, y: number, z: number) {
       this.x = x;
       this.y = y;
       this.z = z;
   }

   add(other: Vec3): Vec3 {
       return new Vec3(
           this.x + other.x,
           this.y + other.y,
           this.z + other.z
       );
   }
}

[MaxGraey]

@liangzeng I think ts -> wasm and optional wasm -> wasm optimization via binaryen

[nidin]

There will be no more features in TurboScript v1 until we get a decent test coverage. TurboScript v2 scope will be like this. emitted code will share common runtime in WebAssembly like malloc, free and stdlib. Unsupported features will emit to JavaScript and imported to WebAssembly. its not really bad because when importing javascript functions to WebAssembly it will generate optimised machine code and hopefully it will be faster, lets see. We will eventually implement these unsupported features in WebAssembly and switch off javascript emission.

[izengliang]

@nidin good idea . c/c++ also can not use webassembly to achieve all features .

javascript + webassembly = c/c++ all features

[izengliang]

The real effect of webassembly is loop and recursion ,so first implement. other language features can be added step by step.

[izengliang]

Typescript ------compile ---> JavascriptWrap + WebAssembly WebAssembly is preview , the way is flexible .

[izengliang]

In the future, can implement the full Typescript features.

Typescript / Javascript ------ compile ---> dynamic (javascript) + static (webassembly)

Feature

Compile life cycle plugins Compile parameters "Use wasm" tag Smart tips Other

☯ Dynamic and static combined into one.

[izengliang]

test.ts

"use wasm";
// compile to javascript
export function test(){}

// compile to wasm
export class User{
     constructor(public name:string){}
}

// compile to js + wasm
export {
    // compile to js
    a(){},

    // compile to wasm
    b(num:number){
       for(let i=0;i<num;i++) {}
    }
} 

test.js

// section compile to js 
module.exports = {
     a(){ },

     b(num){   //  inference is number type 
            "use wasm";   // compile to wasm
             for(;num;num--){ ... }
     }

}

[MaxGraey]

So now we have three ts -> wasm projects with three different approach:

• TurboScript
• speedy.js
• AssemblyScript

[jslegers]

So, let me get this straight...

• You want people to be able to write any code that is either valid TypeScript or valid TurboScript in the same codebase
• If it is valid TurboScript, it will be compiled to WebAssembly
• if it is not valid TurboScript but valid TypeScript, it will be compiled to ordinary JavaScript

Am I correct that this is where you guys are heading towards?

If not, what am I not getting right? If yes, that sounds friggin' awesome!

[nidin]

@jslegers Yes, you got it right. but I don't have any intention to strictly complains with typescript syntax. Master plan is to compile everything to WebAssembly and choose a fast path over typescript but that is an ambitious goal therefore most of the JavaScript features will compile back to JavaScript and step by step implement it in WebAssembly. I am more interested in parallel programming so you can expect multi-threading and SIMD features soon in TurboScript before fancy JavaScript features.

[jslegers]

Ambitious! I'll definitely be keeping an eye on this project...

If you haven't already, you should consider finding yourself a corporate sponsor for this project. I see a lot of potential here, but even the greatest projects tend to whither fairly fast if it's backed only by just a bunch of geeks who do the coding in their spare time.