Listening to new events?

[dantaeyoung]

Is there a way to make an event listener for each new event? getEvents seems like we'd have to poll it.

[parkan]

yep just put an .on('state changed', () => ... onto the biome object 🌴

[parkan]

I could also add a per-event-type emitter (so an event for seed, event for link, etc)

[parkan]

hmm, actually, what is the use case you have in mind?

[dantaeyoung]

The use case is to update the Ore Mill mega viz in real time as things are changing

[parkan]

ok, so due to the previously mentioned conceptual difficulties around event ordering and the commutative nature of CRDTs (and especially arbitrary/custom CRDT support), peer-star-app doesn't provide an API that says "hey this message arrived now", it only says "the state changed, you should look at it"

i.e. it's more properly a state synchronization mechanism than a message bus (which is why at least once delivery semantics are OK here)

I can obviously diff the state on my end and get discrete new message(s) that you need that way, but I wonder if we can just use any state change to redraw the map and have d3 handle the incremental stuff?

[dantaeyoung]

hmmmm, I see.

I think it's helpful to have something between biome and the viz that does the playback of the history to provide current state (with caching, hopefully) and can process and calculate certain things. let's call it a "playback machine".

For example, stats like 'number of peers', or even more important, when people scan their badges at a kiosk, we want to provide a 'state before you scanned' / 'state after you scanned' to provide a good visualization UX to people. (that is, when people scan their badges, they see only the new plants that they hadn't seen before growing)

I think this means that the middle 'playback machine' needs to have a 'get state at timestamp' function, and some caching / new message stuff built inside of it.

maybe it make sense to not put it in as part of biome, then? because that way the playback machine would be doing some work that the biome diffing would already be? what do you think?

[parkan]

hmm do we really need that degree of granularity (full time travel)? I actually explicitly told pedro we don't need time travel 😛

keep in mind that different nodes will have slightly different timelines because messages won't arrive in the same order everywhere, network may partition and heal, etc

'number of peers'

do you mean actual p2p peers in the underlying network, or conceptual peers in the garden link graph? the former has robust tooling and events for in the lib, I can expose those

'state before you scanned' / 'state after you scanned'

I can potentially keep a previous state cache on updates given that I'm copying it (to sort) anyway -- this can either be a diff (these new things were added in this update) or a full copy.

the other thing to keep in mind is that the state change events distinguish between local changes (I changed it) and remote changes (some gossip in the network changed it), so it may not be necessary to do the diffing at all -- if you add something you can just update the viz (on that terminal) with that new event

basically, you can always tell what your action is, for everything else maybe just redraw the world?

[dantaeyoung]

do you mean actual p2p peers in the underlying network, or conceptual peers in the garden link graph? the former has robust tooling and events for in the lib, I can expose those

Oops terminology -- the latter. links that are made between garden neighbors when two people scan their badges together.

basically, you can always tell what your action is, for everything else maybe just redraw the world?

I see. That makes sense, but in the Ore Mill room we might want some more live updates to see what's going on.

I can potentially keep a previous state cache on updates given that I'm copying it (to sort) anyway -- this can either be a diff (these new things were added in this update) or a full copy.

This would be really helpful. After thinking about it, I think I can implement everything if we have this feature.

[parkan]

great ok, I will try to get you this in the first half of tomorrow

would you prefer a delta API ("this is the new stuff") or a .previousState? deltas seems like they would be cleaner because the caches otherwise would need to be attached to events in some unclear way (and have their expiration be managed)

[dantaeyoung]

wow, awesome. deltas would be preferable, awesome, and cleaner!

[parkan]

ok, update: I talked to @pgte and we will try to get real (performant/correct, originating from the CRDT merge operation) diffs into the events for g-sets in time for the event itself 💯

in the meantime, I'll put together an API you can dev against, backed by the hacky approach (keep a prevState copy, diff)