Clojure support?

[bsima]

Are there any plans or interest in using Javelin from Clojure? I was thinking earlier today that it would be nice to have cells on the backend too.

Yeah, core.async works for most of the same problems as Javelin, but the cell abstraction is often easier and simpler to use than channels.

[alandipert]

@stathissideris Hm, I don't see what you do about the relationship between code-walking and cell attachment. In the expression ((formula +) a b),which evaluates to a formula cell, formula is a function, not a macro.

Re: dosync, it's for coordinating changes to input cells, not formula cells -- the values of which can't be set (ignoring lenses). So, assuming add-watch is what's used to detect changes in refs and potentially trigger more propagation if any ref's values have changed, the watches would all be fired after the transaction and its possible retries are completed.

The other required aspect for dosync to work the way I imagine is that formula cells need to be "strict". That is, they need to acquire values as soon as they're created, not when first derefed. Otherwise, propagation would need to happen in the transaction, which does indeed seem bad.

Anyway, it seems like it should work... but you've inspired me to experiment a little because I'm not sure. I'll post whatever I find :+1:

[stathissideris]

@alandipert I realize that formula is a function. I was just trying to implement a syntax similar to your cell= in my implementation, but because I ended up doing it with a function rather than a macro (to avoid code walking), I also have the tradeoff of resolving links at first deref instead of at the time of cell creation. Maybe it's better to accept that the slightly less convenient syntax of formula is preferable to having links be resolved "late".

About propagation: I'm confused, it seems that add-watch is used, but it seems that propagate! is called when doing reset! and it looks like propagate! does take care of formula cells ((if-let [f (x/get (.-thunk next))] (f) (x/get (.-state next)))!?

About dosync: in a multithreaded environment, I think it's an advantage to have swap! and the subsequent propagation of values happen atomically, within the same transaction. Imagine what would happen if you are in the middle of a propagation and another swap! happened on a different input cell. You'd get interleaving propagations happening at the same time, leaving your graph in an inconsistent state. Multiple threads change everything! So I think swap!+propagation should be in a transaction (like in my implementation), and side effects could be in special "output" or "effect" cells.

I'm glad you're feeling inspired! :-)

[stathissideris]

@alandipert Just thought to give you a little demo of the debugging facilities:

(def a (cell 100))
(def b (cell 2))
(def c (formula (partial * 2) a b))
(def d (formula (partial * 10) c))

(print-cells (all-cells))

Prints:

| :id | :label | :formula? | :value | :error | :code | :sinks | :sources |
|-----+--------+-----------+--------+--------+-------+--------+----------|
|   0 |        |     false |    100 |        |       |   #{2} |          |
|   1 |        |     false |      2 |        |       |   #{2} |          |
|   2 |        |      true |    400 |        |       |   #{3} |   #{0 1} |
|   3 |        |      true |   4000 |        |       |        |     #{2} |

[alandipert]

Re: synchronization, I think I understand now, thanks for clarifying. Threads: they change it all!

Re "effect cells", if I understand them as you do, they seem very much like watches, but with the added semantic that they're only called when the dependent cell changes value. As opposed to cells, and more like watches, they wouldn't contain a value and couldn't be depended on directly. If you wanted to retain a value from the effect, you'd swap!/reset! a real cell with it.

So, when these watches are called, the old/new value arguments would always be different. That way you'd have cell-like value dedupe, but all the other properties would be those of watches as they exist already.

Is all of that in-line with what you are thinking?

Re: cell debug - love the table! I can't think of a downside to keeping a list of the cells. An even cooler thing you could do on the JVM is pop a real data grid in a frame, with the ability to edit cell values.

I'm working in a similar area for a work project, based on something I made years ago: http://tailrecursion.com/~alan/ttt/basic.html

We have thousands of cells in the project and it's getting hard to know what's going on, so I'm going to revive this visualization, but include cell names & source file locations when known. I can imagine those would be useful to see in your table also.

[alandipert]

Oops, one difference between cells and watches I overlooked is an obvious one, that cells can depend on many other cells, but watches can depend on only one reference type. So the other feature an "effect cell" would need that watches don't do, is the ability to depend on multiple cells.

I suppose one way to do this would be just to make a formula cell, and then add a watch to it, but maybe there are limitations to that I'm missing. It's definitely less convenient.

[stathissideris]

@alandipert Sorry for the delay, didn't get an email notification and I thought that the thread had died off.

About effect cells: I think you're right in how you've described them.

About the debug table: my interest in cells is because I'd like to make a small ETL/Excel-like JavaFX app, and anything Excel-like needs cells by definition! So I'm thinking of using cells as the overarching metaphor for data processing and the UI as well. Because of this uniformity, I should be able to use the UI both for processing data and for debugging/developing the app itself, so the interactive table that you described is one of the early targets.

Very cool graph visualisation, the fact that it's interactive caught me off guard :-)

I got my cells to the point where all the operations are done in an immutable way (so you can have a graph of cells that's backed by an immutable data structure) and then there are a few thin convenience functions that use the immutable API to mutate the global-cells atom. I hope that having an immutable core will allow me to be more adventurous with the operations that I will support, such as "muting" certain cells (making them "passthrough"). Latest code and tests here:

https://github.com/stathissideris/datacore/blob/62eda26a4e51c3f4c935b02e5d7106831905f292/src/datacore/cells.clj

https://github.com/stathissideris/datacore/blob/62eda26a4e51c3f4c935b02e5d7106831905f292/test/datacore/cells_test.clj

Immutable API shown here:

https://github.com/stathissideris/datacore/blob/62eda26a4e51c3f4c935b02e5d7106831905f292/test/datacore/cells_test.clj#L56-L63

I'm starting to think I should extract this from the app and release it as a separate library.

[dm3]

hey, I've done some work on https://github.com/dm3/javelin/tree/macros-split-clj. It now has a Clojure implementation based loosely on Plumbata. The implementation is not concurrency-safe. All the cell's fields are stored in refs, so on conflicting transactions any watches set on cells might be retried. However, when run from a single thread the implementation passes all of the Javelin tests.

The tests also pass in self-host Clojurescript mode. To try that install Planck or Lumo and run:

planck -c src/:test/:~/.m2/repository/net/cgrand/macrovich/0.2.0/macrovich-0.2.0.jar
> (require '[javelin.core-test :as ct])

The big problem is I can’t figure out how to make the codebase work under normal JVM Clojurescript again!

The only significant change I made was splitting macros out into a separate namespace - javelin.macros. I think the macros can be returned into javelin.core but I’d like to figure out JVM Clojurescript first… Any help greatly appreciated!

[burn2delete]

@dm3 you probably want something like this for the cljs macros

(ns javelin.core (:require [javelin.core :include-macros true]))

[onetom]

A bit more info on this "macros from cljs" topic: https://github.com/clojure/clojurescript/wiki/Differences-from-Clojure#namespaces

In latest versions of the CLJS compiler, you can :require-macros within a cljs NS and when that NS is required by another cljs NS, macros are implicitly available for :referring.

-- tom

[dm3]

@flyboarder @onetom thanks, I'm well aware of the basics. The problems I was struggling with are a bit more nuanced.

I think it's the compilation of macros injavelin.macros which chooses a wrong code walking namespace when in JVM Clojurescript as discussed on Slack with flyboarder. However, I haven't inspected the generated javascript, so the assumption may be wrong.

[jeroenvandijk]

@onetom I saw you asked for a potential use case of implementing cells in pure clojure (cljc for that matter). I'm not sure if i'm being redundant or this has been mentioned elsewhere, but here it comes:

I've been trying to implement a simple version of https://beta.observablehq.com/ in clojurescript. They literally mention the use of cells in their (javascript) implementation. I also had to implement something like cells (i didn't think about Javelin unfortunately) and I found it very convenient to have this cell implementation in pure Clojure so I could run an autotest process on my "cell" logic. So my argument for a pure Clojure version is having a nice test environment! :)

[cddr]

There's an implementation of cells in Clojure now by the original author. It's called matrix.

https://github.com/kennytilton/matrix On Fri, Sep 14, 2018 at 11:20 AM Jeroen van Dijk notifications@github.com wrote:

@onetom I saw you asked for a potential use case of implementing cells in pure clojure (cljc for that matter). I'm not sure if i'm being redundant or this has been mentioned elsewhere, but here it comes:

I've been trying to implement a simple version of https://beta.observablehq.com/ in clojurescript. They literally mention the use of cells in their (javascript) implementation. I also had to implement something like cells (i didn't think about Javelin unfortunately) and I found it very convenient to have this cell implementation in pure Clojure so I could run an autotest process on my "cell" logic. So my argument for a pure Clojure version is having a nice test environment! :)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[jeroenvandijk]

@cddr Thanks! I see the cljs/matrix subdir has a project that only has cljc files :) https://github.com/kennytilton/matrix/tree/master/cljs/matrix

[micha]

I have been working on a JavaFX app and decided to take a stab at porting Javelin and Hoplon to the JVM. I have pretty much everything working now. All the tests pass except for the set-cell! business, which I removed. Code is here: #40. And here is an example of it in action.