Syntax improvements

[AndreVanDelft]

This is a continuation of a mail discussion. The editing in Mac Mail is terrible, so I started this issue.

Shorthand notations can vastly improve readability. In SubScript there is script.., and similar constructs can be made for def, var, val, object, class and case class. For more macro power, with a kind of parameters, I propose that

• a line starting with .. starts a shorthand section
• the shorthand section ends at the first line
  • that has it first token right below or even more to the left of ..
  • and that has the same parentheses level (i.e. not deeper in (), {} or [])
  • and that is neither in a deeper string level
• the first line, possibly with next ones, starts a template
  • next lines are included as long as the nesting level is higher at the line end
  • or as long as a backslash is at the line end, like in C macros
• thereafter follows an instantiation table

The template may contain placeholders of 1 of the following forms (not both!):

• anonymous placeholders; these are by default two or more hash symbols: ##
• named placeholders; these are character sequences between some special quotation delimiters, by default those are backticks. The character sequences should start and end with non-whitespace.
• the table cell separator is by default the vertical bar: |

It is possible to override these placeholder characters, quotation delimiters and cell separators, see later.

Examples:

..
  case class #### [S[_]](####)(implicit val S: Suspended[S],
                                          val F:   Functor[S]) extends ####[S] {
    override def toString = s"####"
  }

  Suspend  | t : () => S[Tree[S]] | Tree   | suspend      
  Call     | t : () => S[Tree[S]] | Tree   | call    
  Sequence | ts:    List[Tree[S]] | Tree   | [*](${ts.mkString(" * ")})                    
  Choice   | ts:    List[Tree[S]] | Tree   | [+](${ts.mkString(" + ")})                
  Success  |                      | Result | 1
  Failure  |                      | Result | 0
  Loop     |                      | Tree   | ...

..
  object #### {
    def apply[S[_]](ts: Tree[S]*)(implicit S: Suspended[S],
                                           F:   Functor[S]): ####[S] = ####[S](ts.toList)
  }
  Sequence | Sequence | Sequence  
  Choice   | Choice   | Choice  

The latter table has some repetitions. Therefore we can add a first row that makes columns in next rows superfluous:

..
  object #### {
    def apply[S[_]](ts: Tree[S]*)(implicit S: Suspended[S],
                                           F:   Functor[S]): ####[S] = ####[S](ts.toList)
  }
  #### | $1 | $1
  Sequence  
  Choice

There we see that only 1 #### placeholder remains, hence the last two rows have only one cell. The other placeholders are resolved,to the value of the first placeholder ($1). We could also do invoke macros here, e.g.,

camelCase($1)

or do a kind of string interpolation, e.g. for prefixing with an underscore:

$"_$1"

Instead of ##### placeholders, we may also have named placeholders. These must be between quotation symbols, by default `. It is possible to override these, e.g., to <<...>>. In case of named placeholders, the instance table gets a header row with the placeholder names, and a separator row with dashes and plusses:

..

  case class `Name` [S[_]](`Params`)(implicit val S: Suspended[S],
                                              val F:   Functor[S]) extends `SuperC`[S] {
    override def toString = s"`Representation`"
  }

  Name     | Params               | SuperC | Representation
  ---------+----------------------+--------+----------------      
  Suspend  | t : () => S[Tree[S]] | Tree   | suspend      
  Call     | t : () => S[Tree[S]] | Tree   | call    
  Sequence | ts:    List[Tree[S]] | Tree   | [*](${ts.mkString(" * ")})                    
  Choice   | ts:    List[Tree[S]] | Tree   | [+](${ts.mkString(" + ")})                
  Success  |                      | Result | 1
  Failure  |                      | Result | 0
  Loop     |                      | Tree   | ...

..

  object `Name` {
    def apply[S[_]](ts: Tree[S]*)(implicit S: Suspended[S],
                                           F:   Functor[S]): `Name`[S] = `Name`[S](ts.toList)
  }

  Name
  ----------
  Sequence  
  Choice

There should be a possibility to do some name manipulation between the placeholder quotes. In the latter object, we reused Name twice. Suppose we wanted to camelCase, or add an underscore, then something like the following would do:

`${camelCase($Name)}`
`_$Name`

Or maybe it should be different. Anyway, it must not be significantly longer.

[AndreVanDelft]

To override placeholder characters, quotation delimiters and cell separators, add a line with the new values, e.g.,

.. ### ` |

This says a an anonymous placeholder has at least 3 dashes. The quotation delimiters are yet again back ticks, and the cell separater is still a bar.

Or:

.. **** << >> ^

This says a an anonymous placeholder has at least 4 asterisks. The quotation delimiters are<< and >>, and the cell separater is ^.

The scope is all following text in the current brace level, or deeper, up to a next overriding.

[AndreVanDelft]

As a test, I rewrite method Tree.rewrite:

  def rewrite: PartialFunction[Tree[S], Tree[S]] = _ match {
    // Call
    case Call(t) => t()

    // Loop
    case Sequence(xs) if xs.contains(Loop()) =>
      def seq: Tree[S] = Sequence( xs.filter(_ != Loop()) :+ new Suspend(S.suspend { seq }) { override def toString = "loopContinuation" } )
      seq
    case Sequence(Sequence(x) :: y) if x.contains(Loop()) =>
      def frag1: List[Tree[S]] = x.filter(_ != Loop())
      def frag2: Tree[S] = Call { () => Sequence(frag1 :+ frag2) }
      Sequence(frag2 +: y)

    // Sequence
    ..
      case Sequence(`Params`) => `Result`

      Params           | Result
      -----------------+---------
      Nil              | Success()
      Sequence(a) :: x | Sequence(a ++ x)
      Call(t)     :: x | Sequence(t() :: x)
      Success()   :: x | Sequence(x)
      Failure()   :: x | Failure()
      Choice(x)   :: y | Choice(x.map { t => Sequence(t :: y) }) // seq-choice distributivity

    // Choice
    ..
      case Choice(`Params`) => `Result`

      Params         | Result
      ---------------+---------
      Nil            | Failure()
      Choice(x) :: y | Choice(x ++ y)  // choice-associativity

    ..
      case Choice(x) if x.`Condition` => `Result`

      Condition                      | Result
      -----------------------------+---------
      contains(Success())            | Success()
      contains(Failure())            | Choice(x.filter(_ != Failure()))
      exists(!resume.isDefinedAt(_)) | Choice(x.map {
                                         case a if !resume.isDefinedAt(a) => rewrite.apply(a)
                                         case a => a
                                       })
  }

Should the cell separators also be given in the last rows of the last table?

[anatoliykmetyuk]

This looks really good.

This shifts the task from the language compilation to DRY representation of plain text. Arguably, this can even be called a human-readable text compression task. Maybe, some techniques from this field can even be applied to measure the effect and justify usage in programming languages.

I am not sure about the table syntax though: this is a sugar on top of the underlying placeholder processing engine and hence should be defined as such. A user should be able to create whatever sugar they like depending on their needs.

IMO there are two different regions of text in such tasks: first, the template, second, the payload for the template. Both needs parsing: the template is parsed to a function Data => String, where Data is template-specific (given the payload, it outputs the ready text); the payload is parsed into a function String => Data (given some text, it outputs the payload that can be used in a template).

I think this engine can be used for any plain-text processing task. Hence, we can not really talk about integrating it with the Scala compiler specifically at this point, since we may want to use it everywhere.

Some questions that will be risen for Scala programmers:

• Integration: how to make sure one can compile your code with a standard set of tools without efforts to find out a particular template you used.
• Error-handling: the compiler will obviously report errors that happen in synthetic sources. Maybe this can be another text processing task, where the error text from the terminal is interpreted, the error is looked up in the synthetic source, and the error line in the synthetic source is translated to a line in the user-editable source?