state of the library

[flip111]

Hello, this library looks cool. What's the state of the library? What's done and what is there still to do? For which version of d3 is it ?

[afcondon]

I had some demo code working - really quite rudimentary but it showed the idea - but then i ran into some problems which at the time i didn't really know how to solve (maybe i know how to solve them now, but i haven't looked at it since then). Principally, IIRC it seemed quite hard to represent the fluid typing that goes between Selections and Transitions and so on.

I'm still very interested in this area and i think that PureScript could be a great language for doing D3-ish things, particularly the much more complicated, interactive things which interested me. As to what it's based on, D3v4 but it was written for the old style row effects in Purescript. That's an easy update to do tho, it was a huge simplification.

i'd be happy to discuss any ideas you have on it - as i say, i still think the idea has legs and the area still interests me.

[flip111]

Can you eloborate on the problems you were facing with the types between selections and transitions?

[afcondon]

I'm afraid i don't remember rn, i'd have to fire it up again and play with it.

the sorts of things that i wanted to do was related to stuff i was (and still am) doing in my day job, involving using D3 (with custom layouts) to layout complex information and manage interaction with it, esp animations and transitions.

In the D3 world which is obviously extremely polymorphic as to types but also numbers of parameters and behaviour of functions given different numbers and types of parameters, the selection remains a sort of thru-line, to which you can at some points add a transition (and it will be a transition from then on) or not (in which case it would just be a selection)

the pseudo-monad kind of function chaining that is the hall mark of D3 led me (and before me the author of an earlier library) to try to mimic that structure as monadic composition, but maybe it really isn't.

with a couple years more experience i now don't really favor these kinds of "tight" wrappers on JS API's...and i've occasionally thought of starting over on this problem.

that said, there is one big advantage to the approach i took here which is the fact that being essentially a DSL for D3 it would be great for converting existing D3 code or for people learning Purescript.

[afcondon]

might be worth adding that what i do now in the work project is to convert the data from "nice" PureScript data types to custom data that is tightly matched to the D3 visualization (which is then thrown over the wall to very clean, no logic, D3 "recipes"). So where you might have NewTypes and ADTs and you could avoid "boolean blindness" with nicer types and use Lists / Foldables / whatever in your Purescript code, you're always just sending over JS arrays, objects, booleans to D3

call backs are a bit hairy but they work too, modulo some hideous tunnelling of the "this" pointer

[flip111]

https://github.com/afcondon/purescript-d3-tagless/blob/master/docs/design%20notes.md#apparent-impossibility-of-typing-transition-and-selection-as-separate-in-any-way

Here you write

In order to chain a transition onto a selection-y monad they have to be the same monad but then you lose the ability to distinguish them as separate types.

Why would you want the ability to have separate types?

---

As i was looking at D3 transitions i noticed something else

https://github.com/d3/d3-transition

Transitions support most selection methods (such as transition.attr and transition.style in place of selection.attr and selection.style), but not all methods are supported; for example, you must append elements or bind data before a transition starts. A transition.remove operator is provided for convenient removal of elements when the transition ends.

So append or bind must be called before transition is valid. So within the monad some functions may not be called before others are called. But my proliminary research into this suggests that the tagless final implementation can not provide such type safety.

[flip111]

Perhaps Indexed Monad can help here. https://stackoverflow.com/questions/28690448/what-is-indexed-monad

It's also said that do-notation is supported with RebindableSyntax https://kseo.github.io/posts/2017-01-12-indexed-monads.html

Which purescript has enabled https://github.com/purescript/documentation/blob/master/language/Differences-from-Haskell.md#extensions

gary made a library for purescript for it https://pursuit.purescript.org/packages/purescript-indexed-monad/1.1.0

[afcondon]

I would love to see a treatment using Indexed Monads for this - whether in the form of a PR or a fork of my attempt here or something done from scratch or even just a gist that illustrates how it would work.

[afcondon]

That’s an excellent idea - i had mentally filed “Indexed Monads” as “something i don’t need to know about right now” but reading Justin Woo’s blog on it, it’s really quite simple and i like that the index shows up in the signature.

i will revisit purescript-d3-tagless with those ideas when i next get a chance.

thanks, A

-- Andrew Condon

On 29 September 2019 at 02:48:35, flip111 (notifications@github.com) wrote:

Perhaps Indexed Monad can help here. https://stackoverflow.com/questions/28690448/what-is-indexed-monad

It's also said that do-notation is supported with RebindableSyntax https://kseo.github.io/posts/2017-01-12-indexed-monads.html

Which purescript has enabled https://github.com/purescript/documentation/blob/master/language/Differences-from-Haskell.md#extensions

gary made a library for purescript for it https://pursuit.purescript.org/packages/purescript-indexed-monad/1.1.0

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/afcondon/purescript-d3-tagless/issues/1?email_source=notifications&email_token=AAJT2X3B6SHRI2NV6LMRSY3QL73OFA5CNFSM4I3OKML2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD73FEQA#issuecomment-536236608, or mute the thread https://github.com/notifications/unsubscribe-auth/AAJT2XZHA6QEL37FXEQX2FDQL73OFANCNFSM4I3OKMLQ .

[flip111]

In case of a new version can it be on D3.js 5 ?

I looked at the source code of this library and i noticed that there is a lot of stuff pulled in from the original D3 purescript library. Mainly i saw most code dealing with low level JS bindings. I wanted to make a quick prototype with indexed monads. But all that extra code that i would have to drag in prevented me from doing so. Do you have any idea how to build up a new version of the code?

Possibly it would be a good idea to drop the dependency on the D3 purescript code. Just copy over what is still valid. So that it's less library code and updated to the new version of D3

[afcondon]

IIRC the leap from v4 to v5 was almost nothing, i’d wager it would even work as is. Certainly when i moved us over at work it was almost a no-op

You’re right about all the boilerplate…especially with the old Purescript row effects.

As i think i probably said before - i would quite like to revisit this in a less slavish “copy the API surface of D3” and try to provide a higher level version of Purescript wrapper….i no longer see much or any value in very close translations of JS APis

-- Andrew Condon

On 9 October 2019 at 18:13:24, flip111 (notifications@github.com) wrote:

In case of a new version can it be on D3.js 5 ?

I looked at the source code of this library and i noticed that there is a lot of stuff pulled in from the original D3 purescript library. Mainly i saw most code dealing with low level JS bindings. I wanted to make a quick prototype with indexed monads. But all that extra code that i would have to drag in prevented me from doing so. Do you have any idea how to build up a new version of the code?

Possibly it would be a good idea to drop the dependency on the D3 purescript code. Just copy over what is still valid. So that it's less library code and updated to the new version of D3

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/afcondon/purescript-d3-tagless/issues/1?email_source=notifications&email_token=AAJT2X32LLPXMZB7XQIARL3QNX7KHA5CNFSM4I3OKML2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAYN4CI#issuecomment-540073481, or mute the thread https://github.com/notifications/unsubscribe-auth/AAJT2XYNRIYDDH3KGUA2THDQNX7KHANCNFSM4I3OKMLQ .

[flip111]

I thought the current implementation of the close translations of the JS API was necessary. Your answer seems to suggest they are not, or at least could be cut down. How do you figure such a reduction can happen? Just by implementing less of D3 or doing the purescript abstraction in another way than before?

By the way about old row effects, are you referring to this? https://purescript-resources.readthedocs.io/en/latest/eff-to-effect.html

[afcondon]

Yes, it was (IMHO) a dead end, at least without much better tooling. I liked (and still like) the idea of breaking out effects in a more granular way than Haskell’s IO monad or Purescript’s Eff…BUT not at the cost of immense unreadable signatures, proliferating effects, incredible amounts of busywork maintaining signatures and compile errors.

It was particularly painful when there were unpleasant semantics about name clashes in rows….

-- Andrew Condon

On 9 October 2019 at 22:47:59, flip111 (notifications@github.com) wrote:

By the way about old row effects, are you referring to this? https://purescript-resources.readthedocs.io/en/latest/eff-to-effect.html

[afcondon]

I think the jury is out on this - need to get something working and get some people using it - i put this library out there and gave some talks on it and so on in the hope that some people would share the vision, but FP is a small community and Purescript is a sub-section of it, so so far that hasn’t happened.

My intuition is that the success of D3 is due to:

• ubiquity of JS as a platform
• rise of data journalism
• high percentage of data journalists who are not programmers
• fantastic, inclusive, development community
• massive library of examples
• comprehensive documentation (see note below tho)

Add to that that Mike Bostock brought huge practical background of data visulization to the project (it’s effectively the third such library he’s written) and you get a project of such unstoppable momentum that the - IMHO

• absolutely terrible software architecture of it doesn’t matter.

TBC it maybe doesn’t matter to anybody but me, i don’t know. But i believe that complex data visualisation and the means to interact with it are a very powerful alternative to building interfaces that are all tables because…we have DBs! and we have tables in HTML! what more do you want!

And if you want to write programs that go beyond one-off widgets, illustrations to articles etc etc, programs which are maintained and extended, programs which have multiple contributors, i think you need more structure, of the kind that Haskell / Purescript provides.

Again, very much IMHO

So…i started out trying to wrap the API of D3 but the futher i got into it the more i felt that it’s just a…i would say bad idea…someone else might just say it’s a particular style…to have highly polymorphic functions which do different things depending on the types and numbers of their parameters, which return different things and have different side-effects based on those things.

It keeps the API surface small but it makes the code extremely complex as evidence by reading the documentation. I think MB’s documentation is almost unsurpassed for an open source project but…it’s necessary because of the aforementioned complexity.

I thought that by mimicking the JS API it would be easier for JS D3 programmers to move D3 scripts to PureScript. I’m now not sure that that is worth it. I think it would be better to make an API that played to the strengths of Purescript and work out how best to map that to D3 as an implementation layer…or directly to the DOM APIs.

TL;DR used to be keener on wrapping the D3 API, now wonder whether it’s worth it

-- Andrew Condon

On 9 October 2019 at 22:47:59, flip111 (notifications@github.com) wrote:

I thought the current implementation of the close translations of the JS API was necessary. Your answer seems to suggest they are not, or at least could be cut down. How do you figure such a reduction can happen? Just by implementing less of D3 or doing the purescript abstraction in another way than before?

[flip111]

I think polymorphic functions in the D3 api are not a problem because one can do monomorphization in the purescript API. Do you have any concrete example of a D3 api surface that you think purescript should represent in a different way?

[afcondon]

I’m thinking of things like transition.attr. Have a look at the docs see what you think:

https://github.com/d3/d3-transition/blob/v1.2.0/README.md#modifying-elements

I’m definitely interested in how you’d represent that.

"# https://github.com/d3/d3-transition/blob/v1.2.0/README.md#transition_attr transition.attr(name, value) <> https://github.com/d3/d3-transition/blob/master/src/transition/attr.js

For each selected element, assigns the attribute tween https://github.com/d3/d3-transition/blob/v1.2.0/README.md#transition_attrTween for the attribute with the specified name to the specified target value. The starting value of the tween is the attribute’s value when the transition starts. The target value may be specified either as a constant or a function. If a function, it is immediately evaluated for each selected element, in order, being passed the current datum d and index i, with the this context as the current DOM element.

If the target value is null, the attribute is removed when the transition starts. Otherwise, an interpolator is chosen based on the type of the target value, using the following algorithm:

1. If value is a number, use interpolateNumber https://github.com/d3/d3-interpolate#interpolateNumber.
2. If value is a color https://github.com/d3/d3-color#color or a string coercible to a color, use interpolateRgb https://github.com/d3/d3-interpolate#interpolateRgb.
3. Use interpolateString https://github.com/d3/d3-interpolate#interpolateString.

To apply a different interpolator, use transition.attrTween https://github.com/d3/d3-transition/blob/v1.2.0/README.md#transition_attrTween ."

-- Andrew Condon

On 10 October 2019 at 12:57:59, flip111 (notifications@github.com) wrote:

I think polymorphic functions in the D3 api are not a problem because one can do monomorphization in the purescript API. Do you have any concrete example of a D3 api surface that you think purescript should represent in a different way?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/afcondon/purescript-d3-tagless/issues/1?email_source=notifications&email_token=AAJT2X5BO3JS5KJFWE5TUZDQN4DDPA5CNFSM4I3OKML2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEA32BLI#issuecomment-540516525, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJT2X5M4G7EAJ2YG2PK3ZTQN4DDPANCNFSM4I3OKMLQ .

[flip111]

Well here it says that a transition can apply to different selections.

https://github.com/d3/d3-transition/blob/v1.2.0/README.md#selection_transition

so my first thought would be data Transition = Transition [Selector]

Then with that selection i suppose transition.attr(name, value) can be called multiple times.

So it becomes data Transition = Transition [Selector] [Attr] with data Attr = Attr Name Value

[afcondon]

Okay, i like data Transition = Transition [Selector] [Attr], but it seems to me to validate my sense that to do a "nice" Haskell-syntax for D3 is necessarily to make something that doesn't look much like D3.js example code. Would you agree?

TBC i'll just repeat what i said earlier in the conversation that i no longer see preserving isomorphism of the JS and Purescript code as a valuable thing.

WRT data Attr = Attr Name Value the problem is that there are so many different types of "value" laid out in that snippet of the docs i pasted above...it's can be null, a constant or a function. Obviously you can say

data Value = Null 
 | ConstantN Number
 | ConstantC Color
 | NumberFn (Datum -> Index -> Number)
 | ColorFn (Datum -> Index -> Color)

or something. Is that what you were thinking?

[flip111]

Would you agree?

I'm thinking about it ... the closest equivalent at the moment would be

d3.select("body")
  .transition()
    .style("background-color", "red");

select :: String -> Selection

transition :: Selection -> Transition

attr :: Transition -> Attr -> Value -> Transition

Now i feel like there should be two types for Transition .. one that has all the infos filled (selectors & attr) and one that is still incomplete. That way you can tell your IndexedMonad that it needs to do another step before executing the JS code (in case of a incomplete transition).

Even so ... whatever types turn out to make the most sense. I still recognize the function names between JS and purescript. So perhaps the more fluid style of D3 gets broken in some parts a bit ... but the intent of the code should still be recognizable.

I hadn't thought about the type for Value, your solution seems good

[afcondon]

i've updated this repo to use spago, purescript 0.13.3, d3v5 etc and minimally tested that it runs in the browser (spago bundle-app -t dist/app.js; open dist/index.html)