README

essence-of-live-coding is a general purpose and type safe live coding framework in Haskell. You can run programs in it, and edit, recompile and reload them while they're running. Internally, the state of the live program is automatically migrated when performing hot code swap.

The library also offers an easy to use FRP interface. It is parametrized by its side effects, separates data flow cleanly from control flow, and allows to develop live programs from reusable, modular components. There are also useful utilities for debugging and quickchecking.

Change the program, keep the state!

Live programs

In essence, a live program consists of a current state, and an effectful state transition, or step function.

data LiveProgram m = forall s . Data s
  => LiveProgram
  { liveState :: s
  , liveStep  :: s -> m s
  }

We execute it by repeatedly calling liveStep and mutating the state. The behaviour of the program is given by the side effects in the monad m.

Example

Here is a simple example program that starts with the state 0, prints its current state each step and increments it by 1:

data State = State { nVisitors :: Int }

simpleProg = LiveProgram { .. }
  liveState = State 0
  liveStep  = \State { .. } -> do
    let nVisitors = nVisitors + 1
    print nVisitors
    return $ State { .. }

We can change the program to e.g. decrement, by replacing the body of the let binding to nVisitors - 1. It's then possible to replace the old program with the new program on the fly, while keeping the state.

Migration

The challenge consists in migrating old state to a new state type. Imagine we would change the state type to:

data State = State
  { nVisitors  :: Int
  , lastAccess :: UTCTime
  }

Clearly, we want to keep the nVisitors field from the previous state, but initialise lastAccess from the initial state of the new program. Both of this is done automatically by a generic function (see LiveCoding.Migrate) of this type:

migrate :: (Data a, Data b) => a -> b -> a

It takes the new initial state a and the old state b and tries to migrate as much as possible from b into the migrated state, using a only wherever necessary to make it typecheck. migrate covers a lot of other common cases, and you can also extend it with user-defined migrations.

Functional Reactive Programming

In bigger programs, we don't want to build all the state into a single type. Instead, we want to build our live programs modularly from reusable components. This is possible with the arrowized FRP (Functional Reactive Programming) interface. The smallest component is a cell (the building block of everything live):

data Cell m a b = forall s . Data s => Cell
  { cellState :: s
  , cellStep  :: s -> a -> m (b, s)
  }

It is like a live program, but it also has inputs and outputs. For example, this cell sums up all its inputs, and outputs the current sum:

sumC :: (Monad m, Num a, Data a) => Cell m a a
sumC = Cell { .. } where
  cellState = 0
  cellStep accum a = return (accum, accum + a)

Using Category, Arrow, ArrowLoop and ArrowChoice, we can compose cells to bigger data flow networks. There is also support for monadic control flow based on exceptions.

Use it and learn it

Setup and GHCi integration

For the full fledged setup, have a look at the gears example, or the tutorial project. The steps are:

• Create a new cabal project containing an executable (preferably in a single file), and add essence-of-live-coding as a dependency.

• There are backend packages available:

  What?	Which backend?	Which library?
  Sound (PCM)	PulseAudio	essence-of-live-coding-pulse
  Sound (Synthesizers)	Vivid & SuperCollider	essence-of-live-coding-vivid
  Sound (Midi)	PortMidi	essence-of-live-coding-PortMidi
  2d vector graphics	gloss	essence-of-live-coding-gloss
  Webserver	WAI	essence-of-live-coding-warp

  Some will require external libraries to link properly. The tutorial project shows you how to install those using Nix.

• There is a custom GHCi script that supply quick commands to start, reload and migrate the program automatically. You can usually copy it from the templates folder.
• Write a main program called liveProgram :: LiveProgram m, where m is a Launchable monad, such as IO.
• In a REPL:
  • Launch cabal repl.
  • Run your program with :livelaunch.
  • Edit your program, and reload with :livereload.
• Instead of reloading manually, you can use ghcid to do this manually for you.
  • Install ghcid.
  • Copy templates/.ghcid into your project folder.
  • Simply launch ghcid and your program will start.
  • Simply edit your file and the changes will reload as soon as they compile.

Examples

• The tutorial project.
• The gears example uses gloss for graphics and PulseAudio for sound.
• There is a demo using the WAI web server demos/app/DemoWai.
• The article contains numerous examples, for example a webserver, a sine generator, and a control flow example
• There are test cases that show how the automatic migration works precisely in certain examples.

Reading

• ICFP 2019 presentation. For a quick pitch.
• Abstract For a 2-page overview.
• Preprint article. For all the details.
• Appendix. For all the details that were left out in the article.
• The dunai and rhine packages. For backup material on how to program in this FRP dialect. (Spoiler: rhine is going towards live coding soon.)

Best practice

In order to get the best out of the automatic migration, it's advisable to follow these patterns:

• Use the FRP interface. (Section 4 in the article, LiveCoding.Cell) It builds up the state type in a modular way that is migratable well.
• Develop modularly. Your top level cell should not be a long arrow expression, but it should call separately defined arrows. This makes the state more migratable, which is useful when you edit only in the "leaves" of your arrow "tree".
• Wherever you write Cells from scratch, or use feedback, use records and algebraic datatypes to structure your state. (Not just tuples and Either.)
• Use ghcid in order to save yourself the hassle of having to reload manually all the time. Vanilla .ghci and .ghcid file templates can be found in the root directory, and usually it suffices to copy these two files into your project and launch ghcid from there.
• When the automatic migration would fail, there are user migrations (LiveCoding.Migrate.Migration). Often, it's good practice to wrap your state in a newtype first (migration to newtypes is automatic), and then migrate this newtype.
• Use exceptions for control flow. (Section 5 in the article) That way, you can keep the control state of your program when migrating.
• Don't keep state in concurrent variables such as MVars, IORefs, and so on, if you don't need to. It cannot be migrated well. (During a migration, the variable might be deleted, garbage collected, and reinitialised.) Instead, all migratable state should be inside Cells. The only use case for such a variable is an external device, resource, or thread, and in this case you should use a LiveCoding.Handle.

Known limitations

• For custom migrations, you currently need to write your own .ghci file.
• If your program doesn't compile when you enter GHCi, you will need to leave GHCi again and recompile. Otherwise the live coding infrastructure will not be loaded. After it has successfully compiled, you can reload from within GHCi, and further compilation errors do not affect the infrastructure.
• The template .ghci and .ghcid files assume that you are using a recent version of cabal (at least version 3). If this is not the case for you, consider updating, or adapt the files to use new-style commands.

Contributors

• Manuel Bärenz https://github.com/turion/
• Miguel Negrão https://github.com/miguel-negrao/