higher order dependencies not being tracked too restrictive?

[ziriax]

I started writing a simple program using React and Sodium, because I am not satisfied with the existing solutions (Redux or Flux feel like imperative state monad programming, the order of reducers determines the effect of an action, and data must be normalized, using identifiers everywhere, ackward and IMHO just as error prone as mutable objects)

The program is a best friend editor, you add persons to a list, and edit their first and last name, and can specify the best friend for each of these persons, by picking from the same list. I wanted something cyclic, a simple todo list is not representative IMO. When a person is removed from the list, all people referring to that person will loose their best friend.

I separated the FRP circuit from the React views, and for each property that can be edited, I added a CellSink. For each event that can occur, for example selecting a person in the list, I added a StreamSink.

I created a higher order lift function that converts every React component into a component that also accepts Cells as props. This works rather nicely, and is strongly typed using Typscript.

However, I use the FRP classes that hold the cells and streams as entities that identify a person, just as in OOP. This means I have a Person class with cell and streams in it, and for selecting a person I have a StreamSink<Person>. The best friend property is a CellSink<Person>

Unfortunately according to the readme, it seems the Typescript version of Sodium does not seem to allow sending Sodium objects to sinks, because it breaks memory management.

Is this also the case when sending a container object holding Sodium objects? Or is it just the case for sending Sodium objects that are not rooted, i.e. Cells and Streams that are created in the fly, like sink.send(new Cell(123))?

It feels very restrictive if my case is not possible , it would mean that I have to introduce unique identifiers and cannot pass around cells and streams through sinks, although these are first class values in Sodium..

However my program does seem to work fine, so either it leaks memory, or I do not understand the limitations correctly.

What exactly are the restrictions regarding memory management in SodiumJS? What are the pitfalls?

PS: I will upload this demo program as soon as possible.

PS: It seems the GHCJS team is tackling this differently, they are actually simulating all GHC features, like threading, stable names, weak references, STM etc using JavaScript. Insane and slow, but it does give more freedom. And hopefully when webasm is stable, this will run fast...

[clinuxrulz]

I think the memory problem is only there if your use of StreamSink or CellSink cause a dependency cycle between sodium object.

I'd imagine though a specialized Sink method could be construct to track the dependency too, similar to the solution they used for lambdas.

E.g. ss.send(value(myObj, [sa, sb]))

Also I think problems occur if Streams/Cells update without a listen. I have not used the TypeScript version myself, but I'm using their same GC in the implementation of the Rust version, with the difference of being able to use the stack for internal reference counting rather than relying on listeners to do that. (I have an easier task)

[the-real-blackh]

@Ziriax, I consider this a very important issue.

Take the case of a cell containing a data structure containing cells. If one of the contained cells is not known to Sodium at the time it is updated, then its updated value can be missed. If you lift the contained cells together, and then switch the result, then everything will work fine, because the combination of the lifts and the switch will inform Sodium of the existence of all the cells. However, if your logic sometimes includes cells and sometimes omits them from the lifting, then there could be a problem with the omitted ones.

I really want to stay on top of this issue. I'd like to look at your code and figure out if the memory management is going to work correctly. So please post your code or if proprietary I'd appreciate it if you could send it to me privately.

It could be that we need to add a way to provide your own function to allow Sodium to find Sodium objects contained in a cell.

[clinuxrulz]

Also the test case https://github.com/SodiumFRP/sodium-typescript/blob/master/src/tests/unit/StreamSink.test.ts#L505 (switchSSimultaneous) touches the same issue. I find its solution a little bit hacky. We should not need to know all possible sodium objects that could be sent through a sink before sending them. We should instead be able to do something to inform the dependency at the point of time of sending them.

[the-real-blackh]

@clinuxrulz Yes, we should look at all the possible ways of doing this and pick some good ones. Looking at all the real-world cases we can find would be a good start. The way I've been approaching it is to think more like a Haskell programmer and consider that the type of the cell implies that there exists is some function to turn a value of that type into a set of streams and cells that are referenced. Then we ask the programmer to supply that function. Another way is, as you say, to add the ability to supply the information at the same time as the value itself.

[ziriax]

I've pushed my current work here.

This is work in progress, so please don't judge the quality of the code yet.

But I don't think it is a problem in my case. The content of the object I'm pushing to the sinks doesn't matter, since it is only used to identify an entity. I could just as well have pushed an identifier of the object, or an index, or whatever.

[clinuxrulz]

@the-real-blackh I intended value(...) to complement lambdaN(...), not replace it. I visualized lamdaN as specifying the dependences of the closure (what the closure captured). Likewise value as what the value captured. value could also be thought if as lambda0.

I feel we should make it dead simple. Such the the end-user can visually see what dependences he/she should provide easily without them thinking too hard. A compiler for a garbage collected language is able to do this by simple inspection of its AST.

[ziriax]

Well, since we can assume the object is immutable, we can just use JavaScript to query all the keys in the object, and figure out the properties that are Cells and Streams, recursively. We can use a WeakMap to cache this. This could be the default behaviour when sending objects that contain new cells and streams? It is actually how part of a garbage collector works more or less, it scans the outgoing references of an object :-)

[clinuxrulz]

Just come back to thinking about this. One field could be added to both Stream and Cell. protected trace_val : ((a: A) => Array<Source>)? = null;

This field could be filled out when we call the trace method on either Cell or Stream (using an immutable update of course), and then the garbage collector in "Vertex.ts" can use that trace_val to look for additional child sodium objects in the contained value. (Which may or may not exist for Stream, depending on if it is currently firing)

Having this feature would allow us to send sodium objects in CellSink / StreamSink, and would also allow us to do a high order loop of cells/streams. E.g. to handle CellLoop<Cell<int>> .

[dakom]

I haven't yet dug into the roots to really understand the nuance - but I appreciate your effort in solving this problem! Thanks!

[clinuxrulz]

Disregard my comment about CellLoop<Cell<int>>, it works fine. And trace is only useful then for CellSink/StreamSink, and I think its too complicated for that. Maybe a send_traced() method or something on both CellSink/StreamSink would be a nicer solution.

[clinuxrulz]

trace can be implemented as a utility using what is already available.

https://gist.github.com/clinuxrulz/6298db3e318db46f6c3e06f5c5eafeeb

Just pipe any Cell containing a value containing sodium objects through cellTrace. This should also work for sending sodium objects via sinks.

[clinuxrulz]

High order dependency tracking is in. Closing for now.