RESTful API to the backend

[wodeni]

Overview

For each client, the backend server is single-threaded and acts as a state machine. However, we would like to make concurrent requests on both the language and graphics sides (e.g. optimizing a diagram while editing Substance program). Therefore, we propose to rewrite the backend server to be stateless and move the server logic for a single client to the frontend. Instead of keeping all states of the client in the backend, each client will store its state in the frontend and make various requests to the backend server with its state as the payload. The frontend is responsible to handle packets in a way akin to the existing state machine:

Potential benefits of this approach include:

• No more synchronization between frontend and backend will be required to keep the overall state of a client. Examples of this include: updating the shapes with label/image dimensions, inferring the value of autostep, and handling the canvas size
• Language services can be offered independently from and concurrently with the optimizer
• New UI features such as optimization history, sharing editable diagrams etc can be supported more easily
• The Haskell modules can be broken down into isolated services and swapped out with alternative implementations if needed
• Simplify the implementation of the state machine since the existing server code can be quite confusing

In this document, we will:

• Provide a specification of the RESTful API supplied by the Haskell backend
• Specify a newer, potentially simplified version of the state machine for the frontend
• Document the expected changes to be made to the existing backend implementation:
  • Serialization of State
  • How and when overallObjFn will be stored and composed
  • Parts of the system influenced of the State rewrite
  • Potential performance costs incurred, if any

Haskell backend API

We specify the backend API by going through all functions used by the existing server module and any other ad hoc computation involved:

Optimizer

• step :: State -> State: take a step in the optimizer

Parsers

• parseDsll :: String -> String -> IO VarEnv: takes in optionally a file name and a program string and returns an Element context (TODO: rename function to parseElement.
• parseStyle :: String -> String -> VarEnv -> IO StyProg: similarly parses a Style program with the Element context as input.
• parseSubstance :: String -> String -> VarEnv -> IO SubOut: parses a Substance program in a similar way; returns SubOut, which is SubProg (VarEnv, SubEnv) LabelMap.
  • TODO: simplify SubOut somehow. We might not want to deal with rerouting this call through the frontend all the time. Also, there can be higher-level functions composed of these parser functions to make the process easier.
• compileStyle :: StyProg -> SubOut -> IO State: takes in the Style AST, substance output and produces an initial State.

Other computation

• In updateShapes: need to change the new shapes to polymorphic and update the translation and state

  let polyShapes = toPolymorphics newShapes
      uninitVals = map G.toTagExpr $ G.shapes2vals polyShapes $ G.uninitializedPaths s
      trans' = G.insertPaths (G.uninitializedPaths s) uninitVals (G.transr s)
      newObjFn = G.genObjfn trans' (G.objFns s) (G.constrFns s) (G.varyingPaths s)
      varyMapNew = G.mkVaryMap (G.varyingPaths s) (G.varyingState s)
      news = s {
          G.shapesr = polyShapes,
          G.varyingState = G.shapes2floats polyShapes varyMapNew $ G.varyingPaths s,
          G.transr = trans',
          G.paramsr = (G.paramsr s) { G.weight = G.initWeight, G.optStatus = G.NewIter, G.overallObjFn = newObjFn }}
      nextClientS = updateState clientState news
      client' = (clientID, conn, nextClientS)

• dragUpdate: updates existing state given drag update

  let (xm', ym') = (r2f xm, r2f ym)
      newShapes  = map (\shape ->
          if getName shape == name
              then dragShape shape xm' ym'
              else shape)
          (G.shapesr s)
      varyMapNew = G.mkVaryMap (G.varyingPaths s) (G.varyingState s)
      news = s { G.shapesr = newShapes,
                 G.varyingState = G.shapes2floats newShapes varyMapNew $ G.varyingPaths s,
                 G.paramsr = (G.paramsr s) { G.weight = G.initWeight, G.optStatus = G.NewIter }}
      nextClientS = updateState clientState news
      client' = (clientID, conn, nextClientSl

• dragShape: logic used to handle drag updates depending on the type of shape

  | shape `is` "Line" =
      trRaw "here"  $ move "startX" dx $
      move "startY" dy $
      move "endX"   dx $
      move "endY"   dy shape
  | shape `is` "Image" =
      move "centerX" dx $
      move "centerY" dy shape
  | shape `is` "Arrow" =
      move "startX" dx $
      move "startY" dy $
      move "endX"   dx $
      move "endY"   dy shape
  | otherwise = setX (FloatV (dx + getX shape)) $ setY (FloatV (dy + getY shape)) shape

• resample: not fully encapsulated logic of resamping the state and re-evaluating the translation

  let news = G.resampleBest G.numStateSamples s
  let (newShapes, _, _) = evalTranslation news
  wsSendFrame conn
      Frame {
          flag = "initial",
          ordering = shapeOrdering news,
          shapes = newShapes
      }

Frontend state machine spec

TODO: discuss with @maxkrieger

Changes needed to the existing system

TODO: discuss with @hypotext and @maxkrieger

• Instead of lambdaizing the overall objective function, we need to store it as data somehow to serialize it. Therefore, there could be a cost of having to re-lambdaize it every time we call step. One concrete thing to check is the cost of this.
• Cost of serialization: would be great if we can profile the cost of serializing various data structures required by the frontend. For instance, VarEnv and State.

[kai-qu]

Looks great so far, thanks for writing this up!

I think we should work out and agree on the type of State in the frontend soon, because that influences the whole design.

For the other computations: I'd say the logic about UI actions should go in the frontend so it can update as fast as possible without making a roundtrip to the server (dragUpdate and dragShape). The other functions that involve changing the translation would likely be better as functions in the backend API (so, updateShapes and resample).

One question: Will multiple clients ever need to interact and share state?

Let me know when/how you want to move this forward. If you are heading out on May 7 then we should schedule a meeting before then.

[maxkrieger]

Will multiple clients ever need to interact and share state?

If they do, I don't think it should happen at the Penrose "layer" of the stack. I.e. if you're referring to realtime collaboration, we should handle that behavior at the same higher-level layer that'll handle the meta-activities like collaborative editors, storing serialized states for sharing, etc. I'm eyeing playing with an Elixir/Phoenix stack (or a hacky Firebase one to start) early on this summer for that layer of the platform.

[wodeni]

Now that State is serializable. @maxkrieger and I will start rearchitecting the system. The first task we agreed on is to write a RESTful version of react-renderer with (1) the WS connection, (2) compressed JSON packets of State, and (3) the following API calls to the backend:

compileTrio 
    :: String -- ^ a Substance program
    -> String -- ^ a Style program
    -> String -- ^ an Element program
    -> Either State CompilerError  -- ^ an initial state generated by the backend (errors will be included in the state somehow)

step 
    :: State  -- ^ the initial state
    -> Int    -- ^ the number of steps n for the optimizer to take
    -> State  -- ^ the resulting state after the optimizer takes n steps

stepUntilConvergence 
    :: State                       -- ^ the initial state
    -> Either State OptimizerError -- ^ the converged state or optimizer errors

[wodeni]

236 addresses this issue. Closing now.