hoplon / javelin

Spreadsheet-like dataflow programming in ClojureScript.
803 stars 44 forks source link

Javelin for the JVM (using refs) #40

Open micha opened 5 years ago

micha commented 5 years ago
This is an implementation of Javelin cells for the JVM. There are no
major differences between the JVM and JS versions, but there are some
minor changes:

* Cells are refs (ie. they extend ARef). So you can use the Clojure
  "dosync", "alter", "ref-set", "ensure", and "io!" with cells. Of
  course, cells still support "swap!" and "reset!" (these are imple-
  mented as calls to "alter" and "ref-set" in a "dosync" transaction
  under the hood).

  Note: In the single-threaded JS environment it wasn't so important
  to use only pure functions as formulas. However, on the JVM trans-
  actions will be retried automatically if there are conflicts, which
  can result in side-effects being performed multiple times.

* Formula cells obtain a read lock on the cells they depend on (ie.
  call "ensure" on their "sources") while updating. This means that
  there will be a conflict and the transaction will be rolled back and
  retried if any of the formula's dependencies are modified by another
  transaction while it is updating.

  In fact, the entire propagation of formula updates is performed in
  a transaction, providing the same consistency guarantees as the JS
  version of Javelin does.

* The cell's formula, watches, and validator will be called asynchro-
  nously, so they are bound to the dynamic environment at create time
  (eg. see "clojure.core/bound-fn"). So the dynamic bindings of the
  thread where the watch is added will be available to the watch
  function when it is called later.

* Watches only fire if the cell's value has changed. The normal ref
  behavior is to fire watches after every update.

* Cells hold weak references to the cells that depend on them so they
  may be garbage collected when they become unreachable. Cells in JS
  hold strong references which prevent cells from being disposed of
  until both the cell itself, and all of the cells its formula depends
  on, have become unreachable.

  Note 1: Formulas will continue to update and watches will continue
  to fire until the cells are destroyed by the garbage collector.

  Note 2: Dangling weak references are not cleaned up. I'm not sure
  what the best approach is, yet. But the WeakReference objects don't
  consume a lot of resources --- seems OK to leave it for now.

* The "formula-of" and "formulet" macros are now aliased to "fn=" and
  "let=", respectively, to save some typing and also it looks cool.

* Cells can be mutated (ie. input cells converted to formula cells and
  vice versa, or the formula and/or sources of a formula cell updated)
  using the "cell!" and "formula!" functions. The "cell!=", "fn!=",
  and "let!=" macros (with aliases as above) are also provided.
xificurC commented 5 years ago

Any chance this gets merged and released?

joinr commented 3 years ago

I have resuscitated this recently for a project that's using javelin for incremental computation (scoring functions and constraints for local search for discrete optimization). I incorporated this PR into master, but there were only minor changes to build.boot and the readme....is there any reason this PR has not been accepted?

burn2delete commented 3 years ago

@joinr I don't believe this has been battle tested. However I'm happy to merge this PR and push a release if you believe it to be sufficient for your production use case.

xificurC commented 3 years ago

A release with a warning in the readme sounds reasonable to me, at least people can easily give it a spin and provide feedback/issues/PRs

joinr commented 3 years ago

@flyboarder I'm actually going through and trying to unify under cljc after merging. The tests all pass as well, so while not quite "battle tested" there certainly isn't a regression so far. There appears to be a slight disconnect between the implementation for javelin.cell/cell-doseq and the implementation in javelin.core/cell-doseq (from core.cljs), although I think they should probably do the same thing. So, right not, none of the examples that use the function cell will work with the clojure implementation out of the box (since cell lives in javelin.cell, not javelin.core). I think getting this cleaned up is logical, although it doesn't stop merging the PR outright.

joinr commented 3 years ago

I'm curious about some of the jvm stuff too; like the comments regarding circular dependencies causing runtime looping. It seems like testing for cycles is fundamental to dataflow (e.g. excel does this naturally), and is pretty easy to implement (or extract from an existing lib). Curious if that's just a limitation of the jvm backend or if there isn't any validation of the DAG during propagation that indicates cycles (somewhat off topic).

burn2delete commented 3 years ago

@joinr Currently there is no cyclic validation of the DAG, runtime loops are not prevented