Ideas for next version of ts2fable

alfonsogarciacaro commented 7 years ago

Pinging @jgrund @MangelMaxime @mike-morr @fsoikin. If you want to collaborate please tell me and I'll make you collaborators of this repo :)

These are some of the ideas I have for the next ts2fable version. Please have a look and share your feedback in the comments.

Convert the code to Typescript. The main reason to use TS instead of Fable is that we need to use intensively the Typescript compiler API so it makes sense to take advantage of the samples, documentation, etc already in TS, as well as the autocompletion provided by VS Code.
Use Typescript type checker: The current ts2fable version just does syntax parsing, the info provided from the type checker would make it easier to check inheritance, etc. It also makes it easier to translate the comments.
Use F# 4.1 recursive namespace: This is very easy, just add rec after namespace on top of the file and we don't have to worry about writing and for all the types. And nested modules can also reference modules below them.
Use F# functions (int->int->string) instead of delegates (System.Func<int,int,string>) for the signatures. Delegates are not needed in Fable 1.0, as F# lambdas don't compile as nested functions any more.
Put all the types of a TS module in an F# module suffixed with _types, including the generated Globals type to hold module functions and values. The actual module will just become a reference to the Globals type, more or less as it's now in the node bindings.
Only use interfaces, no classes. Classes are cumbersome in the binding files because you need to fill the body with the jsNative placeholder. Also many features of Typescript classes cannot be represented in F# (for example, an interface in TS can inherit a class). In order to represent a class we will create two interfaces: one for the instance members and another for the static members. Then we'll add a reference to the static interface in the Globals type. Example:

class Foo {
   constructor();
   myProperty: number;
   myMethod(i: number): void;
   static myStaticMethods(s: string): number;
}

type Foo =
    abstract myProperty: float with get, set
    abstract myMethod: float -> unit

type FooStatic =
    [<Emit("new $0($1...)">]
    abstract Create: unit->Foo
    abstract myStaticMethods: string -> float

type Globals =
    abstract Foo: FooStatic

alfonsogarciacaro commented 7 years ago

Conventions to keep from current ts2fable:

Erased unions with U2, U3.... Now with the !^ handling them should be easier for users.
Constructor and callable interfaces: use Crete and Invoke with Emit attribute.
StringEnum, etc...

fsoikin commented 7 years ago

On the subject of converting to TypeScript... May I suggest using this tool? I originally made it with C# in mind, but it's template-based, and so flexible enough to produce anything. It uses the typechecker, so can handle inferred types and JS source, etc. And it's been working quite well for my company for years now.

Some caveats: it is in very much a beta stage, uses a lot of hacks and suboptimal approaches. But the core is pretty solid (and unit-tested ;-). I am just at the moment working on repackaging it like an adult, and while I'm at it porting it to the latest TypeScript, and adding support for some missing features (such as union types).

As for collaboration: I wouldn't mind, except I don't quite know what it means. :-)

jgrund commented 7 years ago

@alfonsogarciacaro This looks good to me; I'd like to help as well.

One thing I noticed is that it's nice to pass in Records instead of Class instances for methods / constructors that take a POJO config.

MangelMaxime commented 7 years ago

@alfonsogarciacaro Using TypeScript is a really good idea.

Neftedollar commented 7 years ago

@alfonsogarciacaro why don't you implement ts2fable with F# parsers?(FParsec for example)

alfonsogarciacaro commented 7 years ago

@Neftedollar I don't think this is a good idea, because it would mean doing a lot of work for something that the Typescript compiler API gives you for free. The hard part of ts2fable is not parsing .ts files into an AST (this is done by the TS compiler) but representing TS idioms in F#.

Neftedollar commented 7 years ago

@alfonsogarciacaro thanks for explaining

MangelMaxime commented 7 years ago

One more important thing to do with the next version of ts2fable is writing a doc explaining the binding structure (default case, special files etc.)

For example, sometimes it's not only a question of generating the bindings but also the user need to write an helper file (like React, Snabbdom, etc.).

Also, if there is some edge cases we should explain how to solve them. I am thinking here about the translation of globals function (attached to window in the browser) which is often a problem with the current ts2fable.

This should help us having a standard structure for a Fable binding.

alfonsogarciacaro commented 7 years ago

@jgrund Yes, I think we can use POJO Records for specific cases. However, in many cases the TS interfaces to represent these option objects have many optional fields, and F# record constructors force you to instantiate all the fields, which can be cumbersome.

@fsoikin That's really interesting, thanks a lot for the link! Yes, it would be really nice to use your project and take advantage of the synergies :) I had a quick look at the documentation and the sample but I couldn't get a clear idea of how to put everything together yet. Would it be possible to create a very simple F# sample for reference? Also you mentioned union types are not supported, what happens in this case? Are they replaced by any or just ignored?

mike-morr commented 7 years ago

@alfonsogarciacaro I am brand new to F#, and I am still trying to get up to speed. I was working on converting ts2fable to TypeScript, but the issue is that without Intellisense, it is hard to figure out what the types are supposed to be since they changed the API. For that reason, I started exploring other options before rewriting it.

Kotlin

One that looked interesting, was trying to leverage ts2kt. This is the Kotlin equivalent of ts2fable. It is currently using TS 2.0, which gets us a lot closer to the latest. I believe the jump from 2.0 - 2.2 would be much less painful than coming from 1.8 to 2.0. The reason it looked interesting to me is that the Kotlin syntax looks like something that could be easier to convert to F#, and they already have the 2.0 TS Compiler typed to Kotlin See Here. They recently released JavaScript as a compile target, I guess it has been experimental for a long time.

Anyways, what I wanted to explore was either seeing if we could convert those compiler declarations from Kotlin to F#, which would enable intellisense for the TS compiler in F#. The other idea, was just forking their code and using Kotlin for ts2fable and modifying it to output F# instead of Kotlin. Jetbrains/Kotlin seems to be going through the pain of the scenario you were trying to avoid. i.e. They are using Kotlin to parse the TS and output Kotlin. If we could somehow piggy back on that project to get a better and/or faster result, I think that would be ideal.

Contribute

I would love to contribute. I am fairly strong on TypeScript, but weak on F#. I learn pretty quick though and prior experience with C# has been helpful with learning F#. I started looking at F# for the first time seriously 3 days ago and in combination with Fable it was love at first site. That being said, I think contributing to this project would be a great learning exercise and would love to contribute and learn from you and the community along the way if you're cool with it.

Proposed Roadmap Items

Angular

Angular 2 support. i.e. Requires decorators and a few other challenges. I believe in about 1 -2 years, Angular usage will be >= React. There is a ton of Angular 1 projects out there especially in the enterprise. Migrating to React would be insanely painful, but Angular 2+ has an interop story and some of the concepts and terminology are the same. Also, Angular should interop with Elmish very well. It already supports Redux.

DefinitelyTyped

I would like to see us using definitely-typed as a test bed. Make sure we can convert the top 200 downloaded declarations. That repo also has tests for the declarations, so maybe we could use those somehow.

TypeScript Agility

The other thing I think is critical is having a way to keep ts2fable closely tracking TS. One of the weaknesses of ts2fable right now, is I don't think it is capable of converting any features that were added past 1.8. I would guess at this point most projects have moved beyond that. Right now, I feel like people are sort of pigeon holed into using React with Fable for SPA scenarios.

Cli Integration

I think a killer feature would also be adding functionality to the dotnet fable cli to download @types/anylibrary and have it converted and added to the project automagically. So the dev experience would be almost identical to TypeScript.

Fable

It would be awesome if we could add package features to the CLI. Think sugar on top of NPM where we could try to do what elm does by hard enforcing semantic versioning.

Conclusion

Sorry for the novel, but I wanted to get all of this out there. I love this project and would love to help.

Disclaimer

Since I am still new to this and learning, It is very possible that I made some suggestions above that don't make sense or are unrealistic.

alfonsogarciacaro commented 7 years ago

@mike-morr Thanks a lot for sharing those insightful comments! I need to check about the Typescript bindings (I assume you already tried npm i @types/typescript). I had a similar problem in the past, and had to download the latest declaration files directly from the repo (they're automatically generated for every build I think) and then reference them manually with a /// <reference... comment (don't remember the syntax exactly). But then you need to be careful not to use bindings newer than the typescript version you have installed in package.json.

Kotlin

Yes, I had a look at the Kotlin parser when writing ts2fable for the first time, but at that moment the project seemed to not be maintained. As you say, it seems to be more active now so I need to check again. However, it should provide a very big benefit to compensate the layer it puts in between Typescript and F#. I still hope we can use the TS compiler API directly.

Angular

I'd like to support angular too. It's a bit tricky with decorators because .NET is very restrictive with attribute arguments. There was a brief discussion about this in the past in https://github.com/fable-compiler/Fable/issues/347, maybe we can revive that. Or I think there's also a plain JS API without decorators, we could target that as well.

DefinitelyTyped

Yes, this is the way to go for sure. FunScript had something similar (in a time when TypeScript had many less features) And finding a way to translate the tests will make the Fable bindings much more robust indeed.

Cli Integration

Definitely, it'd be very awesome if we could automate more tasks in CLI commands like acquiring bindings and publishing packages. We also need to find a way to avoid having to type all Fable dependencies (probably reading them directly from the .fsproj).

jgrund commented 7 years ago

@alfonsogarciacaro

Yes, I think we can use POJO Records for specific cases. However, in many cases the TS interfaces to represent these option objects have many optional fields, and F# record constructors force you to instantiate all the fields, which can be cumbersome.

I just saw the keyValueList fn for the 1.0 beta. Is that the preferred way to work with config objs?

MangelMaxime commented 7 years ago

@jgrund I think yes it the preferred way now. At least if you all the config property are optional is easier to use it than a POJO. Because for a POJO you would need to fill all the property not only the one you need.

alfonsogarciacaro commented 7 years ago

@jgrund @MangelMaxime Yes and no ;) For manually edited helpers (like React, Fetch) is the preferred way for the reasons commented by @MangelMaxime

For automatically generated I'm not sure, for the moment i think just an interface should be enough. Although it's a bit more verbose they can be instantiated in the following way:

let opts = createEmpty<MyLibOptions> // This is just an interface in the bindings
opts.foo <- Some 5  // Note this is typed
opts.bar <- Some "hi"
MyLib.start(opts)

MangelMaxime commented 7 years ago

@alfonsogarciacaro Interface looks good too. :) And as you said probably easier to generate from the ts definition

fsoikin commented 7 years ago

@alfonsogarciacaro sorry for not responding for so long, been a bit swamped.

There is an example config under /examples/fsharp. To run it, pull the repo (be sure to check out the vnext branch), build it (it's under FAKE), and then run:

node ./built/src/tstc.js -c ./examples/fsharp/.tstconfig ./examples/module.ts

This will produce output in /examples/module.fs.

There is a caveat though, which I haven't realized before: grouping and ordering.
C# pretty much just concatenates all source files together and then compiles it all, which makes the order and grouping of stuff insignificant.
F#, on the other hand, requires that types in one module are defined all together, can't be interspersed with types form other modules.
The current core of this tool doesn't support control over grouping and ordering. I have some thoughts on how it can be implemented, but I can't give an ETA.

Same thing with union types (and, I'm sure, some other newer TS features): not hard to add support, but I'm not sure when I'll be able to.

The bottom line is, the tool would potentially work, but not in its current form, so I can't really advocate for using it now. Your call.

MangelMaxime commented 7 years ago

@alfonsogarciacaro I started using the interface version for experimentation. And I am wandering can't we remove the option flag on the members Like so, your example become:

let opts = createEmpty<MyLibOptions> // This is just an interface in the bindings
opts.foo <- 5  // Note this is typed
opts.bar <- "hi"
MyLib.start(opts)

This is a little less verbose. I think the option flag come from the d.ts and can help telling to the user you don't need to set all the properties. But I find the Some keyword doing noisy code here.

alfonsogarciacaro commented 7 years ago

@fsoikin Great, thanks! I'll try to check next week. Maybe the ordering problem can be solved with the F# 4.1 recursive namespaces namespace rec Foo, but again I'd need to check.

@MangelMaxime This is tricky. F# has no notion of optional members as Typescript, so we have to use members with optional type. I know this is verbose, but in these cases although it's easy for a human to understand when an interface is used for options and when not, automatise it is more difficult. In the above sample, let's say you want to check whether the member foo is defined or not, if it's an option type you can do:

match opts.foo with
| Some i -> ... // Fable translates this as a non-strict null checking
| None -> ...   // so it works with undefined too

However if the foo member is just an int, you cannot do this. With string and other types you could check agains null, but not for numbers. So there you have a JS pattern that you cannot represent in F#.

For simplicity, I would keep translation optional members as option types in F#. The equivalence of JS null/undefined with F# None is working well.

The other alternative would be to consider optional members as just nullables, so with strings and other interfaces we don't need to to anything but numbers would have to be defined explicitly as Nullable and probably we would have to implement the nullable operators. The main problem with this is nullables are also very verbose to define in F#:

opts.foo <- Nullable(5)

MangelMaxime commented 7 years ago

@alfonsogarciacaro Ok you convince me :)

ctaggart commented 7 years ago

I think that ts2fable should be written using Fable. Today I posted about using the TypeScript Compiler API from Fable. https://twitter.com/cmr0n/status/919919033234935808

You can both read and write the TypeScript API from Fable. It would be amazing if we could generate the F# code using the FSharp.Compiler.Service with Fable. Anyone know the possibility of doing so? https://github.com/fsharp/FSharp.Compiler.Service/issues/825

alfonsogarciacaro commented 7 years ago

That's awesome @ctaggart, thanks a lot for this! ❤️ I've added some more comments in your PR :)

About generating F# AST, unfortunately I think it's not very easy. Fable processes the typed AST provided by FCS, but even if the cases for this resemble the Quotation API they're all just active patterns. The actual constructors are much more complicated and IIRC they're not exposed, so it's not possible to generate F# AST the same way you can do it with Quotation constructors (as FunScript did).

For the simple purposes of ts2fable (generate type signatures and no actual code) it's probably enough to just use strings to emit F# code, but that's just an opinion :)

ctaggart commented 7 years ago

With #19 merged into the develop branch, it addresses several of these issues. I'd like to see this issue closed and any remaining issues separated out so that they can be discussed and addressed. Separated issues to be addressed very soon:

https://github.com/fable-compiler/ts2fable/issues/20 Use F# functions instead of delegates
https://github.com/fable-compiler/ts2fable/issues/21 map imports

alfonsogarciacaro commented 7 years ago

You're right, @ctaggart. This issue is too broad, let's deal with individual tasks, thanks a lot for opening the new issues 😄

fable-compiler / ts2fable