Braces revisited

[AndreVanDelft]

Currently there are several types of brace pairs that separate the process part in scripts from Scala code. Normal braces for normal atomic actions, and braces with immediately next to them one of these symbols: * ? ! . ....

I wanted to make it more difficult to specify normal atomic actions, for these are rather CPU expensive, so I thought these should be specified using {! ... !}. These are now in use for tiny code fragments, and I thought those could use the plain braces instead.

Anyway, braces for tiny code fragments should also become less needed: method calls as tiny code may also use the plain script call syntax, and assignments could use the let construct (work in progress). BTW currently a method-with-script-call-syntax yields a normal atomic action; this should become a tiny code fragment.

But now I think tiny code fragments should be within {: ... :}; every kind of code fragment would then have - no kind would be the default. The main reason for this proposal is that braces would have a use to mark Scala statements (resulting in a value) that would get an implicit conversion.

A use case is the data store proxy sample application. Its live script would be:

live = << req: InformationRequest ==> {dataStore ? req} 
                     ~~(data:Data)~~> {dataStore ? DetailsRequest(data)}
               ~~(details:Details)~~> {!  sender  ! (data, details) !}
       >> 

{dataStore ? req} etc. yield futures, which require implicit conversions to scripts.

BTW I had in mind to write a different kind of brace pair: {=dataStore ? req=} but this does not really appeal to me.

[AndreVanDelft]

I am now writing a submission for the Spash - Onward conference. I will apply this new braces proposal.

[AndreVanDelft]

All these sophisticated brace variations may look distracting, and that is not necessarily a bad thing. I think their use should be discouraged: there are alternatives for tiny code fragments; normal atomic actions should be used sparsely, foremost in library scripts. The {*...*} pair for background tasks will stand out nicely.

[AndreVanDelft]

BTW So {dataStore ? req} could be implicitly converted to a future-conversion script. Likewise other pieces of Scala code such as {sender ! (data, details) } could be implicitily converted to a tiny code fragment script, so that for tiny code the nice normal braces could be used. Maybe this would even be possible through an implicit script macro, but note that annotation code must know at some point in time the type of there.

[anatoliykmetyuk]

Technically, the result of {dataStore ? req} is Future[Any]. So we'll need some wrapper around the script result values logic, so that this wrapper converts the Future into a Script node (or any other node) "on the fly".

[AndreVanDelft]

I think the wrapper would be an implicit script, just like the ones for buttons and characters in the LookupFrame example. The braces just enclose a value item, rather than an atomic action; this would require a minor change in the parser.

[AndreVanDelft]

Some more thinking. In the current situation, when parsing script expressions: {} always enclose Scala code () are for parameter lists and for scoping behavior operators in script expressions; [] are for script lambda's

In Scala code () are for parameter lists and for scoping of operators in value expressons.

We need also some way to denote value expressions in Scripts. In particular for tuples. E.g. against Slick 3.0 we would like to write for a query q on a database db:

(db,q) ~~(s)~~> println(s"value is $s")

The parser should see the first part as a tuple, to which an implicit conversion would apply, but the parser should anyway also expect a parenthesized behaviour expression.

So the parentheses symbols are too heavy overloaded. Let's stop that. IMO we should stop using them as scope delimiters of behaviour operators. We could easily take [] instead; they are already used for script lambda's when in the context of Scala code, and this new use would be rather logical.

As soon as we will reach a stable state we could change this in the new compiler. Then all example programs may need some rewriting. The old compiler should IMO also be updated with such a change, IMO; it is relatively simple, and the compiler would be needed FTTB as a backup.

[anatoliykmetyuk]

I don't think we should replace () by [] as a scope delimiter. () as a scope delimiter is intuitive, it is used like this everywhere. If we replace it, this will create a barrier for the users.

Parser can differentiate between (foo, bar) as a tuple and (foo) as a value expression: if it sees the commas, it is a tuple, otherwise - an expression.

[AndreVanDelft]

Smalltalk and Objective-C use [] already. For the reasons, see these statements by Brad Cox and Tom Love. IMO, once the Parboiled parser becomes stable we should experiment with this.

[anatoliykmetyuk]

Do they use them for scoping? We can experiment, of course, but this would be yet another bit of knowledge for the users to learn. Many will find themselves frustrated when things won't work the intuitive way.

[AndreVanDelft]

Use case: the example Twitter search controller. Old code was:

    liveScript = initialize ; (mainSequence / ..)...

    initialize = view.main(Array())

    mainSequence = anyEvent(view.searchField)
                   waitForDelay
                   searchTweets ~~(ts:Seq[Tweet])~~> updateTweetsView(ts)
                              +~/~(t: Throwable )~~> setErrorMsg(t)

    waitForDelay = {* view.setStatus("waiting"  ); sleep(keyTypeDelay) *}
    searchTweets = {* view.setStatus("searching"); twitter.search(view.searchField.text, tweetsCount)*}

    updateTweetsView(ts: Seq[Tweet]) = @gui: {view.setTweets(ts)}
    setErrorMsg     (t : Throwable ) = @gui: {view.setError(t.toString)}

New version:

    liveScript = initialize ; [mainSequence / ..]...

    initialize = view.main: Array()

    mainSequence = anyEvent: view.searchField
                   waitForDelay
                   searchTweets ~~(ts:Seq[Tweet])~~> updateTweetsView: ts
                              +~/~(t: Throwable )~~> setErrorMsg: t

    waitForDelay = {* view.setStatus("waiting"  ); sleep(keyTypeDelay) *}
    searchTweets = {* view.setStatus("searching"); twitter.search(view.searchField.text, tweetsCount)*}

    updateTweetsView(ts: Seq[Tweet]) = @gui: view.setTweets: ts
    setErrorMsg     (t : Throwable ) = @gui: view.setError: t.toString

Changes:

• liveScript has now rectangular brackets.
• parameter lists are now without parentheses, but with a colon instead
• Inside updateTweetsView and setErrorMsg there are no more braces; view.setTweets: ts becomes a tiny code fragment

[AndreVanDelft]

Another use case: several lines from LookupFrame2. Old version:

  implicit script vkey(??k: Key.Value) = vkey2(top, ??k)

  script..

    doExit            =   exitCommand @gui: {confirmExit} ~~(r:Boolean)~~> while (!r)

    searchSequence    = guard(searchTF, ()=> !searchTF.text.isEmpty) 
                        searchCommand
                        showSearchingText searchInDatabase showSearchResults / cancelSearch

    showSearchingText = @gui: {outputTA.text = "Searching: "+searchTF.text}
    showCanceledText  = @gui: {outputTA.text = "Searching Canceled"}
    showSearchResults = @gui: {outputTA.text = "Results: 1, 2, 3" }

    progressMonitor   = ... @gui:{outputTA.text+=here.pass} {*sleep(200)*}

New version:

  implicit script vkey(??k: Key.Value) = vkey2: top, ??k

  script..

    doExit            =   exitCommand @gui: {!confirmExit!} ~~(r:Boolean)~~> while: !r

    searchSequence    = guard: searchTF, (()=> !searchTF.text.isEmpty) 
                        searchCommand
                        showSearchingText searchInDatabase showSearchResults / cancelSearch

    showSearchingText = @gui: {outputTA.text = "Searching: "+searchTF.text}
    showCanceledText  = @gui: {outputTA.text = "Searching Canceled"}
    showSearchResults = @gui: {outputTA.text = "Results: 1, 2, 3" }

    progressMonitor   = ... @gui: {outputTA.text+=here.pass} {*sleep(200)*}

Atomic actions are now marked with exclamation marks, as in {!confirmExit!}. Inside doExit, !r should be accepted as a simple value expression. Otherwise we would have to write while: (!r) or stick with while(!r) The actions inside showSearchingText etc became silently tiny. For Scala code we could in principle allow ":" style parameter lists as well, but FTTB I skip that.

[AndreVanDelft]

At LambdaConf in Boulder I presented this example:

class DataProxy(dataStore: ActorRef) extends SubScriptActor {

  script live = 
    << req: InformationRequest ==> {dataStore ? req} 
                  ~~(data:Data)~~> {dataStore ? DetailsRequest(data)}
            ~~(details:Details)~~> {  sender  ! (data, details)}
    >> 
    ...
}

The problem is that this does not work; e.g. {dataStore ? req} is a code fragment, whereas it should be taken as a value, ready for implicit conversion (in this case to a future). Now we could say that we would not need parentheses this way: bare braces could always denote value expressions that require implicit conversion (and have {: :} and {! !} for tiny code fragments and atomic actions). There are two objections: first I find { } nicer for tiny code fragments instead of {: :}. Second, if we would start using { } for Scala values, then IMO we should do the same as in Scala: both () and { } yield values; the difference being that the latter may contain sequential operators etc.

So IMO it seems more logical to reserve ( ) for value expressions and parameter lists. Then rectangular brackets [] would be required in script expressions; and these would mean about the same as currently inside native Scala code.

  script live = 
    << req: InformationRequest ==> (dataStore ? req)
                  ~~(data:Data)~~> (dataStore ? DetailsRequest(data))
            ~~(details:Details)~~> {  sender  ! (data, details)}
    >> 
    ...

[AndreVanDelft]

And there is the example that I posted in this thread a few weeks ago:

(db,q) ~~(s)~~> println(s"value is $s")

Specifying a tuple there would require this new proposed use of the parentheses.

[AndreVanDelft]

From the file downloader example:

    selectFile = var selectionResult: FileChooser.Result.Value = null
                 @gui: {selectionResult = fileChooser.showSaveDialog(destinationFileLabel)}
                 if selectionResult != FileChooser.Result.Approve then break else
                 @gui: {destinationFileField.text = fileChooser.selectedFile.getAbsolutePath}

    downloadSequence = val url = new URL(sourceUrlField.text)
                       val destinationFile = new File(destinationFileField.text)
                       var is: BufferedInputStream = new BufferedInputStream(url.openStream())
                       var size: Int = is.available()

                       val os: BufferedOutputStream = new BufferedOutputStream(new FileOutputStream(destinationFile))

                       @gui: {
                         progressBar.max   = size
                         progressBar.min   = 0
                         progressBar.value = 0
                       }

                       val chunkSize = 8 * 1024
                       var read      = -1
                       var processed = 0

                       (
                         var buff: Array[Byte] = new Array[Byte](chunkSize)
                         {* read = is read buff *}

                         if read != -1 then (
                            {* os.write(buff, 0, read) *}
                            {processed += read}
                         )
                         @gui: {progressBar.value = processed}

                         while (read != -1)
                       )

                       {
                         is.close()
                         os.flush()
                         os.close()
                       }

This code has been modified a little from the version in the repository; IMO this should be parseable. In the new approach the code would become:

    selectFile = var selectionResult: FileChooser.Result.Value = null
                 @gui: let selectionResult = fileChooser.showSaveDialog: destinationFileLabel
                 if selectionResult != FileChooser.Result.Approve then break 
                 else @gui: let destinationFileField.text = fileChooser.selectedFile.getAbsolutePath

    downloadSequence = val url = new URL(sourceUrlField.text)
                       val destinationFile = new File(destinationFileField.text)
                       var is = new BufferedInputStream(url.openStream())
                       var size = is.available()

                       val os = new BufferedOutputStream(new FileOutputStream(destinationFile))

                       @gui: {
                         progressBar.max   = size
                         progressBar.min   = 0
                         progressBar.value = 0
                       }

                       val chunkSize = 8 * 1024
                       var read      = -1
                       var processed = 0

                       [
                         var buff = new Array[Byte](chunkSize)
                         {* read = is read buff *}

                         if read != -1 then [
                            {* os.write(buff, 0, read) *}
                            let processed += read
                         ]
                         @gui: let progressBar.value = processed

                         while (read != -1)
                       ]

                       {
                         is.close()
                         os.flush()
                         os.close()
                       }

Here two [] bracket pairs replace parentheses. Some val and var declarations are now simpler, without specifying the type explicitly. The let construct makes some braces disappear. It announces a tiny code fragment; just like val and var to be ended at the next semicolon, or, if it is earlier, at end of the line provided that the code fragment could end there syntactically.

[AndreVanDelft]

From LifeFrame:

      boardControl     = ...; noise / (.. singleStep) multiStep || clear || randomize

      do1Step          = {*board.calculateGeneration*} @gui: {!board.validate!}

      noise            = 'n'; ... @gui: {board.doRandomize()} {*sleep*}
      randomize        =   randomizeCommand @gui: {!board.doRandomize()!}
      clear            =       clearCommand @gui: {!board.doClear!}
      singleStep       =        stepCommand do1Step
       multiStep       = multiStepStartCmd; ... do1Step {*sleep*} 
                       / multiStepStopCmd

      speedControl     = ...; speedKeyInput + speedButtonInput + speedSliderInput

    setSpeed(s: Int)   = @gui: {!setSpeedValue(s)!}

      speedKeyInput    = while(here.pass<10)
                       + ( val c:Int = pass_up1+'0'; key(chr(c)) setSpeed(char2Value(c))) // TBD: make here an implicit parameter

   speedButtonInput = (if speed>minSpeed then speedDecButton)
                    + (if speed<maxSpeed then speedIncButton)

     speedDecButton = minSpeedButton setSpeed(minSpeed) + slowerButton setSpeed(speed-1)
     speedIncButton = maxSpeedButton setSpeed(maxSpeed) + fasterButton setSpeed(speed+1)

   speedSliderInput = speedSlider setSpeed,speedSlider.value

      mouseInput    = (mouseClickInput & mouseDragInput)
                      /  doubleClick (mouseMoveInput / doubleClick {!resetLastMousePos!})
                      ...

   mouseClickInput  = var p:java.awt.Point=null
                      mouseSingleClick( board, ?p) 
                        ...
                      ; {! resetLastMousePos !}
                      ; ( {*sleep_ms(220)*} {!here.break_up(2)!} / doubleClick )
                      ; ...
                      {! handleMouseSingleClick(p) !}
                      ...

   doubleClick      = var p:java.awt.Point=null; mouseDoubleClick(board, ?p)
   mouse_Released   = var p:java.awt.Point=null; mouseReleased(   board, ?p)
    mouseDragInput  = mouseDraggings(board, (e: MouseEvent) => handleMouseDrag(e.point)) / (mouse_Released  {!resetLastMousePos!}); ...
    mouseMoveInput  = mouseMoves(    board, (e: MouseEvent) => handleMouseMove(e.point)) 

New:

      boardControl     = ...; noise / [.. singleStep] multiStep || clear || randomize

      do1Step          = {*board.calculateGeneration*} @gui: board.validate

      noise            = 'n'; ... @gui: board.doRandomize {*sleep*}
      randomize        =   randomizeCommand @gui: board.doRandomize
      clear            =       clearCommand @gui: board.doClear
      singleStep       =        stepCommand do1Step
       multiStep       = multiStepStartCmd; ... do1Step {*sleep*} 
                       / multiStepStopCmd

      speedControl     = ...; speedKeyInput + speedButtonInput + speedSliderInput

    setSpeed(s: Int)   = @gui: setSpeedValue: s

      speedKeyInput    = while(pass<10)
                       + [ val c = (pass_up1+'0').toChar; key: c setSpeed: char2Value: c ]

Intermezzo: in the latter line there is a nice conversion of the computed integer to a character; only possibly by the new meaning of the parentheses. Another pecularity: nested calls with ":" parameters.

   speedButtonInput = [ if speed>minSpeed then speedDecButton ]
                    + [ if speed<maxSpeed then speedIncButton ]

Intermezzo 2: It would be possible to write this without the brackets, using the prefix notation for the plus:

   speedButtonInput =+ if speed>minSpeed then speedDecButton
                       if speed<maxSpeed then speedIncButton

     speedDecButton = minSpeedButton setSpeed: minSpeed + slowerButton setSpeed: (speed-1)
     speedIncButton = maxSpeedButton setSpeed: maxSpeed + fasterButton setSpeed: (speed+1)

   speedSliderInput = speedSlider setSpeed: speedSlider.value

      mouseInput    = [ mouseClickInput & mouseDragInput ]
                      /  doubleClick [mouseMoveInput / doubleClick resetLastMousePos]
                      ...

   mouseClickInput  = singleClick
                      ~~(p:Point)~~> 
                      [ resetLastMousePos {*sleep_ms(220)*} handleMouseSingleClick: p 
                      / doubleClick
                      ] 
                      ...

   singleClick      = mouseSingleClick: board
   doubleClick      = mouseDoubleClick: board
   mouse_Released   = mouseReleased: board
   mouseMoveInput   = mouseMoves:     board, ((e: MouseEvent) => handleMouseMove(e.point)) 
   mouseDragInput   = mouseDraggings: board, ((e: MouseEvent) => handleMouseDrag(e.point)) 
                      / mouse_Released resetLastMousePos
                     ...

This assumes mouseSingleClick, singleClick etc have a java.awt.Point as result. Now mouseDragInput and mouseMoveInput have each an ugly parameter. This could become better using the stream flow operator:

   mouse_Move     = mouseMove: board
   mouse_Drag      = mouseDrag: board
   mouseMoveInput = [mouse_Move..] ~~(e:MouseEvent)~~>> handleMouseMove: e.point
   mouseDragInput = [mouse_Drag..] ~~(e:MouseEvent)~~>> handleMouseDrag: e.point
                    / mouse_Released resetLastMousePos
                    ...

Note that mouse_Move etc return a MouseEvent. After each success of mouse_Drag, [mouse_Drag..] succeeds, yielding the same MouseEvent. This is then handled.

It seems that section 2.9 of the ArXiv'd paper "Some New Directions for ACP Research" is flawed. It introduced a new operator to support the stream flow arrow ~~~>>. This auxiliary %/%/ operator was said to "perform x with the modification that each time after an atomic action in x happens, y happens". The main problem is that x does not yield a value if it does not succeed; the other problem is that when x contains a loop, each with a sequence of actions, then probably one would like the y only to be done after each loop pass; not after each action.

A better informal meaning of %/%/ would therefore be: "perform x with the modification that each time that x succeeds whereas it can also continue with a one or more actions, y happens before x resumes with these actions; when x finishes successfully then y happens."

[AndreVanDelft]

In my enthusiasm I forgot that the form

mouseMoves:     board, ((e: MouseEvent) => handleMouseMove(e.point))

was created this way because of its efficiency: it installs a mouse listener one time only, which repeatedly invokes the call back function. The efficiency is about the same as plain Scala code.

Let's make a dataflow based alternative that is more efficient than the one above.

Suppose mouseMoves consists again of a looping event handling code fragment {... ...}; but does not have a call back parameter any more; rather the script succeeds on each mouse event. Then we could write:

   mouse_Moves     = mouseMoves: board
   mouseMoveInput = mouse_Moves ~~(e:MouseEvent)~~>> handleMouseMove: e.point

The left hand side of the data flow arrow is more efficient than the right hand side, since it does not need to be reactivated each time any more. Could we do the same for the RHS?

In fact we only want to map the result value of type MouseEvent to handleMouseMove: e.point. For result value mapping I had the following syntax in mind:

  term "^{" scalaCode "}" 

However, something similar to the current dataflow operators would be preferable. E.g.,

  term "~~(p:Param)~^{scalaCode}"

would map the result value of termp as p to scalaCode. In case the result should be forced to become the result value of the enclosing script then write:

  term "~~(p:Param)~^{scalaCode}^"

or simply some code using the _ placeholder:

  term "~~^{_+1)}^"

All this may be implemented efficiently using a wapper for the term:

ScriptWrapper(r: Script[R], mapFunction: R => S) = ???

The RHS would be a Script that overrides $result to mapFunction(r.$result)

I am not happy with the syntax yet. OTOH such a map capability would give much of the power of Parboiled2.

[AndreVanDelft]

The last few lines of the previous post are incorrect. ScriptWrapper should be a class that extends Script; internally it holds another script (i.e. a script expression; not necessary a script call). Now whenever that wrapped script attempts to set a result value at a higher level, this passes through the mapFunction. I don't know exactly yet how this should be programmed; I just feel it may be done.

[anatoliykmetyuk]

The new syntax has been implemented and works correctly, except the stream dataflow operator.

There are many ways to organize the lhs and rhs lifecycle: each can be activated only once or can be reactivated for every piece of data being sent. While for the lhs it makes sense to activate it only once, rhs must be activated for every new piece of data if we want to support arbitrary script expressions as an rhs. If we assume rhs to always be a scala code expression, it can also be activated only once and then reused.

There are also several ways to organize the execution procedure for the stream dataflow either.

• The most efficient way is for ~~~~>> to bind the rhs to lhs so that rhs is called as an ordinary Scala code from the lhs which is an eventhandling loop. This is the same way it is done in Swing's Scripts.scala, but with the task to be executed being abstracted out. However, it will imply that lhs should always be an eventhandling loop: no way to use a script or an annotated loop as an lhs.
• Another way is to send every piece of data emitted by the lhs upwards the graph with a message (AAHappened or another). We can create a separate graph node for the ~~~~>> operator, a binary operator node. In its Continuation handler (Continuation will be sent each time AAHappened or another message with the data arrives to the node), we'll consume the piece of data, generate the template of the rhs based on that data, activate it and execute. The drawback here is that the speed of data transmission will be less than desirable, since the data is passed through irrelevant nodes via the SubScript messages before making it to the ~~~~>> node. Also if a stream does some heavy data transmission this will slow down the entire SubScript VM, since it'll have too many data messages to handle.
• Yet another way is to create a VM dedicated to the stream dataflow handling and run it in a separate thread. It will operate on a graph similar to SubScript VM, but only the nodes relevant to the stream will be spawned there. For example, in @foo: {... send(x) ...} ~~(x: Int)~~>> println(x), the annotation node will not appear in the stream VM. The data will be transmitted through the graph using messages. This means that the data messages will pass only through the relevant nodes that can change their behaviour and further path. This also implies a great deal of control over the messages: we can play with various stream graph nodes to organize different flows of data. The communication with the SubScript VM will be done via callbacks, like the threaded code blocks do when they notify it when they have finished their execution.

Since there are so many strategies for implementing the stream dataflow, I'll wait until you can express your opinion on which one is optimal. Meanwhile, I'll look at the task of the new website and SubScript JS.

[AndreVanDelft]

It is indeed wise to proceed now with the web site and the JS version. If you had not already done so, you could implement the syntax for the stream dataflow operator. The implementation would just do nothing.

FTTB I do not want to follow any of your 3 implementation strategies for this operator. I think it can be done in a more ACP’ish way. But I appreciate the fact that you wrote down these alternatives; they may become useful anyway.

In the paper “Some new directions for ACP research” I have described in a few words how the stream dataflow operator would work. Note that I made an update to that paper, which relates to this operator; the second version of the paper is the appropriate one.

The definition would require a "mandatory interruptions” operator: x %/%/ y

Some rules:

Upon activation this activates x Upon a success of x, y is activated. When both x and y have success, the whole has success. Same for deactivation. Upon an action happening in y, x is suspended (in case it had still unsuspended activated actions) Upon an action happening in x, y is deactivated (in case it had activated actions) Upon a success of y, a new "iteration pass” of x is activated or resumed, if it had already been activated

It is a bit more complicated than this, because the operator is n-ary. I do not see much use for 3 or more operands; the operator is mainly meant to fill the impedance gap between Scala streams and scripts.

This operator, like a few others that start with ‘%’, require a suspend-resume mechanism. Suspend is a bit like Exclude, but it does not cause leave deactivations; it just suspends leaves. I.e. it may set a flag, which makes sure such atomic actions will not be executed FTTB. Resume does the opposite. And there are the OnSuspend/OnResume hooks that need to be called at appropriate times.

I think I can work on this when I am back home.