Run LiveScript in ES6 Harmony mode - let and generators support

[kristianmandrup]

Hi,

I love LiveScript and recently started playing around with ES6. Been using the generators branch since yesterday, but I'm greatly missing being able to use let. Why not have LiveScript support a --harmony option similar to node, where the new generators syntax is available and where let is the default over var? If someone could just show me how to ghet started and point me in the right direction I would love to dig in and try to deliver this feature for livescript. What do you think?

[igl]

Generators are being worked on (in a branch) and the let keyword is unfortunately used for anonymous functions:

a = let
  1 + 1

[vendethiel]

anonymous functions:

Rather, IIFEs.

[kristianmandrup]

But why not have the --harmony option generate let variables by default:

x = 1 becomes let x = 1. Then you can override with explicit var declaration such as: var x =1 This way it would not conflict with the livescript let keyword for anonymous functions.

The livescript OOP syntax also easily support generation to the new ES6 class syntax. Almost the same, except the initializer is named: constructor.

Why not start a harmony branch, branching off the generators branch ;) Cheers!

[qgustavor]

Why not directives/pragmas? "use harmony" maybe, so will be possible to compile code which uses ES6 along with code that does not.

I just don't know if it's hard to implement, but it can be used in other ideas: maybe giving LiveScript compatibility with the ES6 of operator (which is swapped with the in operator).

[kristianmandrup]

From some investigation, I found that the following changes would be useful for some basic ES6 usage with let

lexer

Add "let" as valid DECL

https://github.com/gkz/LiveScript/blob/master/src/lexer.ls#L175

case \export \const \var then tag = \DECL

Change to:

case \export \const \var \let then tag = \DECL

Remove "let" keyword, use "own"

https://github.com/gkz/LiveScript/blob/master/src/lexer.ls#L191

case \let \own

Change to:

case \own

Default let

Matches an identifying literal: variables, keywords, accessors, etc.

https://github.com/gkz/LiveScript/blob/master/src/lexer.ls#L115

try Function "var #id" catch then @carp "invalid identifier \"#id\""

Change to:

try Function "let #id" catch then @carp "invalid identifier \"#id\""

language

Add let as declaration in language?

https://github.com/gkz/LiveScript/blob/master/src/lang-ls.ls#L58

[\var // ^ #ident //]

Add:

[\let // ^ #ident //]

grammar

https://github.com/gkz/LiveScript/blob/master/src/grammar.ls#L56

: o \ID -> Chain L Var $1

In the comments it states Expression MATH Expression

I guess if the Var expression evaluates to let we are all set ;) Can't figure out where the Var expression is defined?

If we want to introduce a Let expression, where do we do this?

: o \ID -> Chain L Let $1

[igl]

Judgeing from this comment (https://github.com/gkz/LiveScript/issues/481#issuecomment-40262143). No generators for 1.3 :/

ES6 is still a long way to go and probably will only be fully implemented in LS2.(Don't take my word for that though)

And to be honest: All those things are not really important to me as a LiveScript-coder. More compile time checks (like const, := and maybe soon for types) would be way way way nicer than all those experimental, unoptimized es6 features which will have to be stubbed/transpiled anyway for the next 5 IE versions.

[vendethiel]

Judgeing from this comment (#481 (comment)). No generators for 1.3 :/

This comment seems to exactly indicate that generators are planned for 1.3, though ?

[kristianmandrup]

Why not directives/pragmas? "use harmony" "

Yes, I think that would be the most elegant solution. Any idea how to achieve this? As I see it, changes I needed in multiple files. You would need to have 2 versions of LiveScript, one LiveScript-ES5 and a LiveScript-ES6. The the main file parser should select the LiveScript-ES6 when it detects a "use harmony" directive at the top of the file, right?

[kristianmandrup]

Who cares about IE? and I think many developers use LiveScript with node on the server side (I know I do) = no IE :)

[apaleslimghost]

I'm -1 on pragmas. "use strict" is only a pragma because you don't compile Javascript.

I'm definitely of the opinion that Livescript should be able to run wherever Javascript can, and any feature that generates ES3-incompatible syntax should be behind a compiler flag (or at least have a compiler flag to turn off).

[kristianmandrup]

https://github.com/gkz/LiveScript/blob/master/src/command.ls#L2

require! {  
  '..': LiveScript  
  '../es6': LiveScript-es6
  path  
  fs  
  util  
  'prelude-ls': {each, break-list}:
  prelude  './options': {parse: parse-options, generate-help}
}
version = LiveScript.VERSION
es6-version = LiveScript-es6.VERSION

https://github.com/gkz/LiveScript/blob/master/src/command.ls#L79

!function compile-script filename, input, base  
  options = {filename, o.bare, o.const}  
  t       = {input, options}  
  try    LiveScript.emit 'lex' t

Should detect if there is a "use harmony" directive, then set the local livescript := LiveScript-es6 and proceed... sweet :)

!function detect-livescript-es filename
  # read first 5 lines and see if directive "use harmony" can be found there
  harmony-directive = fshoot 'readFile' source, !-> detect-directive source, ... ??
  if harmony-directive? then LiveScript-es6 else LiveScript

!function compile-script filename, input, base
  var livescript = detect-livescript-es filename
  options = {filename, o.bare, o.const}  
  t       = {input, options}  
  try    livescript.emit 'lex' t

Something like this could work I guess? (draft proposal)

[igl]

Ah yes nami.. I confused they are actually talking about the sourcemap branch there, not the generator branch. So yes there probably will be generators for 1.3 :)

[vendethiel]

So yes there probably will be generators for 1.3 :)

That's not a "maybe" :-).

[kristianmandrup]

@quarterto I agree, directive is not the best way, compiler flag feels better/safer, but directive could be left as an option. Since you might want to run in es6 mode on a source tree (say via grunt or WebStorm file watcher?), but you might have special cases within which you want it to treat specifically (code ported from some other place f.ex). Not a perfect world we live in so we gotta stay flexible IMO.

[kristianmandrup]

https://github.com/gkz/LiveScript/blob/master/src/lang-ls.ls#L44

Add support for ES6 modules

| o[frn] | off | return | break | and | let | var | loop | yes | export | import | default

more to follow...

[kristianmandrup]

I have now spent quite a few days working on this: https://github.com/kristianmandrup/LiveScript/tree/ES6

So far I have managed to set up the basic infrastructure. Now my only problem is how to tweak the lexer/parser to produce ES6 compatible javascript.

For starters I want to add lock scoping with let. Does anyone have any idea where to make such a tweak? For me it's like looking for a needle in a haystack!

The current problem is that for var, it somehow detects if whether it has already been defined. If not it adds the variable to the list of variables for that scope, to be added in the var list for that scope. Let should not do this as I understand it.

[kristianmandrup]

Okay, I get it. I shouldn't look in the Lexer but in the Grammar.

    o 'Chain ASSIGN Expression'
    , -> Assign $1.unwrap!, $3           , $2
    o 'Chain ASSIGN INDENT ArgList OptComma DEDENT'
    , -> Assign $1.unwrap!, Arr.maybe($4), $2

But where do I find documentation on how to use this Grammar language and what these statements mean?

Assign $1.unwrap! Huh? Should I checkout the Jison repo? Couldn't find anything in the Jison Docs. And what about this statement?

# Our handy DSL for Jison grammar generation, thanks to
# [Tim Caswell](http://github.com/creationix)

Browsed through @creationix repos, but couldn't find anything that sounded like it related to jison or bison!? And no dependencies in package.json to a DSL that I can see.

In jscore.jison

AssignmentExpr
    : ConditionalExpr
    | LeftHandSideExpr AssignmentOperator AssignmentExpr

So I assume Assign somehow references this Expr? And what about $1.unwrap?? Please help me understand... thanks!

[vendethiel]

Jison is here. $N just refers to the (N+1)th param in the first argument, so for both of those $1 is Chain and $2 is ASSIGN.

[kristianmandrup]

Yes, I understand that part. This is written as a comment at the top. But what about Assign $1.unwrap! and such? I cloned the whole jison repo to have a reference when I get "deep down". Thanks a lot @Nami-Doc

[vendethiel]

For the first part, it's one of our AST Node. For $1.unwrap, well, since $1 is an instance of Assign, the code is here

[kristianmandrup]

Awesome! I'm sure that will help me enormously! As long as I know how to track the code I can go deeper down the rabbit hole. Thanks again :)

[kristianmandrup]

# __Chain__ can be unwrapped as its inner node, if there are no subnodes.
  unwrap: -> if @tails.length then this else @head

but no @tails method/property in the AST, only prototype.children = ['head', 'tails']; So I guess head and tails relates to the node in the AST (tree), tails being the children and head the parent node and this being the current node. Clever!

[kristianmandrup]

And I guess then that Assign $1.unwrap!, $3, $2 creates an instance of the Assign class.. and then? How does it result in the id added to the var list and how can I avoid it for let !? Too hardcore for me just yet...!

[vendethiel]

Scope itself is declared here -- at the bottom. You can find its methods declare, add etc.

Assign.tails is assigned here. You need to check the constructors to see the properties :)

[vendethiel]

Really, these" questions are better for IRC -- #livescript@irc.freenode.net :)

[kristianmandrup]

Agreed! Thanks :) See you on IRC. I downloaded Colloquy, Connected to irc.freenode.net and searched for #livescript room to join, but no such room found.

[kristianmandrup]

My current ideas... not quite as easy/simple as I had imagined!

case \export \const \var \let then tag = \DECL

https://github.com/gkz/LiveScript/blob/master/src/ast.ls#L2527

exports.Decl = (type, nodes, lno) ->
  throw SyntaxError "empty #type on line #lno" unless nodes.0
  DECLS[type] nodes

DECLS =
  export: ...
  import: ...
  const: ...
  var: Vars 
  # TODO
  let: LetVars

#### Var
# Variables.
class exports.Var extends Atom
  (@value) ~>
  ::isAssignable = ::isCallable = YES
  assigns: -> it is @value
  maybeKey: -> Key(@value) <<< {@line}
  varName: ::show
  compile: (o) -> if @temp then o.scope.free @value else @value

## or could we add a @type to Var instead?
#### LetVar
# LetVariables.
class exports.LetVar extends Atom
  (@value) ~>
  ::isAssignable = ::isCallable = YES
  assigns: -> it is @value
  maybeKey: -> Key(@value) <<< {@line}
  varName: ::show
  compile: (o) -> if @temp then o.scope.free @value else @value

# Declares uninitialized variables.
class exports.Vars extends Node
  (@vars) ~>
  children: [\vars]
  makeReturn: THIS
  compile: (o, level) ->
    for {value}:v in @vars
      v.carp 'invalid variable declaration' unless v instanceof Var # also Let unless exception?
      v.carp "redeclaration of \"#value\"" if o.scope.check value
      o.scope.declare value, v

# Declares uninitialized let variables.
class exports.LetVars extends Node
  (@vars) ~>
  children: [\vars]
  makeReturn: THIS
  compile: (o, level) ->
    for {value}:v in @vars
      v.carp 'invalid let variable declaration' unless v instanceof Let
      v.carp "redeclaration of \"#value\"" if o.scope.check value
      o.scope.declare value, v

!function Scope @parent, @shared
  @variables = {}
  @let-variables = {} # is this a good idea?

  # Adds a new variable or overrides an existing one.
  add: (name, type, node) ->
    if node and t = @variables"#name."
      if @READ_ONLY[t] or @READ_ONLY[type]
        node.carp "redeclaration of #that \"#name\""
      else if t is type is \arg
        node.carp "duplicate parameter \"#name\""
      else if t is \upvar
        node.carp "accidental shadow of \"#name\""
    # Dot-suffix to bypass `Object::` members
    # should we store let vars in separate list?
    switch type
    | \let  => @let-variables"#name." = type
    | otherwise => @variables"#name." = type
    name

# Declares a variable unless declared already.
# let is the new default!
  declare: (name, node) ->
    if @shared
      return if @check name, void, type # pass type also
      scope = that
    else
      scope = this
    type = if constant then \const else @type-of node
    scope.add name, type, node

  # TODO: we need a LetVar class!
  type-of: (node) ->
     return \var if v instanceof Var 
     return \let if v instanceof LetVar 
     node.carp "Unknown variable type #v"    

  # Ensures that an assignment is made at the top of this scope.
  assign: (name, value) -> @add name, {value}

  # Checks to see if a variable has already been declared.
  # Walks up the scope if `above` flag is specified.
  check: (name, above, type) ->
    vtype = if type is \let @let-variables"#name." else @variables"#name."
    return vtype if vtype or not above
    @parent?check name, above unless type is \let

[vendethiel]

Ah, it's seems it's "chat.freenode.net".

[kristianmandrup]

Yes, Im on there now. What is your nick there? Im @kmandrup

[kristianmandrup]

Still no-one else is interested enough in ES6 support to help implement it?

How about backtick support to escape code from being compiled as livescript, similar to this solution?

http://stackoverflow.com/questions/21096717/preferred-way-of-working-with-es6-modules-and-coffeescript

MyComponent = Ember.Component.extend
  classNames: ['pretty-color']
  attributeBindings: ['style']
  style: (->
    "color: #{@get('name')};"
  ).property('name')

`export default MyComponent`

[vendethiel]

This works, you just need more backticks :p. WRT ES6 at hand, I'm starting to consider it – at least in a branch – but I'm not the one making decisions.

[kristianmandrup]

more? please show above example in livescript :) double backticks on each side?

Ah yes, that is exactly what is says on the API doc page for livescript :) double up!

"Change any JavaScript code literals, eg. `js code here` to ``js code here``"

[kristianmandrup]

Why not support the export/import ES6 module syntax in the version 1.3? Ember.js now uses this mechanism by default and Angular 2.0 will use it too... I'm sure most JS frameworks are heading in this direction

[duralog]

I'm not gonna say anything dangerous here, however, if you put in your package.json: "LiveScript": "duralog/AliveScript" what you have requested will indeed work properly. (it's my personal derivative and not a full branch... for my personal use ... you're welcome to fork, whatever it)...

cheers

[vendethiel]

So now, people are forking forks of forks ? I feel like we failed at some point ;-).

[gkz]

We have generators and yield now. The two things that we want now are let statements and JS for .. of loops, for working with iterators. Those are tricky though because they conflict with existing syntax.

[kristianmandrup]

What about ES6 module syntax support!? That seems to be the most used (early adapted) feature of web frameworks such as Ember... I guess back ticks will do for now, but not ideal

[gkz]

Yes, the module syntax is also an important part. It seems easier than let and for ... of because fewer people use export and import currently.

Maybe there should be a backwards incompatible LiveScript 2.0 that isn't a complete rewrite, but still has major updates like these.

[Delapouite]

Maybe there should be a backwards incompatible LiveScript 2.0 that isn't a complete rewrite, but still has major updates like these.

Absolutely. That's a solid plan.

[apaleslimghost]

If we're removing the import operator we should add an import function to Prelude.

[Delapouite]

My understanding is that only import as an alias will be reclaimed. The corresponding operators <<< and <<<< will remain in place.

[apaleslimghost]

Actually, I'd much rather get rid of all the ASCII-barf operators...

[vendethiel]

^

[kristianmandrup]

Please, let's get this out as Livescript 2.0 before Xmas with support for generators, yield, and export/import modules :) Remove the current import alias for sure. Thanks :)

[kristianmandrup]

Move any conflicting keywords to prelude and let's have LiveScript 2.0 implement most of the ES6 specs. We could just use ES6 shims with $ functions like forOf$ etc.

Many of the shims can be loaded via npm, then simply include a given shim function the first time the expression is encountered and call that function. Next time simply call it again (same approach as used now). Why not? Most frameworks as already using ES6 via transpilers now, but I prefer the macro approach https://github.com/jlongster/es6-macros which LiveScript would pretty much provide as well. Cheers!

    "es6-shim": "x",
    "es5-ext": "x",
    "es6-map": "x",
    "es6-set": "x",
    "es6-symbol": "x",
    "es6-iterator": "x",
    "es6-template-strings": "x" 

[AprilArcus]

Hey folks; just wanted to note a great potential use case for this sort of thing: if LiveScript could compile its class syntax to ES6 classes, I could pipe the resulting output through Babel with @gaearon's babel-plugin-react-transform and thereby take advantage of hot-loading React components.

[ozra]

Strong +1 for pragma route over compile flag - if possible without exploding the compiler.

[ozra]

@vendethiel - regardng forks.. well, LS is a fork of Coco, is a fork of CS, isn't it? I'm happy that happened ;-) (Might be confused on the matter though)

[vendethiel]

Forks all the way ;-).

[igl]

Just merge elm into LS! :balloon: