Higher level bindings for d3 using r2d3

[riccardopinosio]

Hello,

This package looks awesome and it promises to solve a lot of my pains with including customs d3 scripts in Rmarkdown, so thanks for the work on this! I have a question on whether you plan to include higher level R bindings for d3 visuals in the package. If I understand it correctly now the package allows one to include a js d3 script seamlessly in R markdown/shiny, however one still needs to code the visual in d3 itself, which is not that easy for beginners. I thought it would be awesome to provide higher level R bindings to the d3 API so that plots can be done entirely in R, e.g. in the context of htmlwidgets. Any plans on moving in this direction? If not, I might take up this challenge myself, if you think it would add value.

Best! Riccardo.

[jjallaire]

What you are talking about pretty much describes many htmlwidgets (higher level R syntax for D3). I think what you are suggesting however might be an R syntax that emits D3 code/visualizations. That is certainly an interesting idea but not one we are taking up right now (my biggest concern about that idea is that since all of the documentation/examples/SO posts/etc. on D3 are in JavaScript that R users might actually be better off learning D3 than using an R API which has more limited exposure/support.

[riccardopinosio]

Indeed, there's various htmlwidgets which expose (quite limited) parts of the d3 API to R users. But I was thinking about something relatively more ambitious, that is, really providing something like the ggpot2 API feel with d3 under the hood, so with a serious investment on creating conceptual mappings from R to d3.; in some sense this would be in the same vein as ggvis.

I also had your concerns about whether such a tool would actually be useful, so you can perhaps help me in clarifying my ideas here. I can code in d3 pretty much in my sleep by now and so I myself would just rather use your r2d3 package and my awesomely maintained .js scripts.

However, isn't the whole idea behind something like e.g. ggvis to provide an R (tidyverse) syntax that emits js visualizations, with the whole song and dance of pipes, tidy eval etc.? If that idea is useful, then, one could also think about providing the same with d3 under the hood using the r2d3 package rather than vega (ok, vega is declarative so the API mappings would probably be easier). If I look at myself introspectively, even though I use javascript every day, I would rather like having a package that integrates seamlessly with my tidyverse code and allows me to produce d3 visuals with an R tidyverse syntax. I suppose this would be even more true of an R programmer who doesn't know javascript.

I guess it boils down to deciding whether such an R syntax would add relevant "syntactic" and "conceptual" sugar that would make it worthwile to use in place of just coding d3 yourself. But couldn't you ask the same question of e.g. ggvis, why don't you just use vega in the first place?

[jjallaire]

You make some excellent points here! I think it would indeed be quite valuable if R users could use tidyverse constructs/syntax to code D3. I agree it's ambitious but it would be very cool if it was done well!

[riccardopinosio]

I see. Then I will try to get a start on a more general tidyverse API to d3 visuals that uses r2d3 under the hood; as I said, it would be of quite some use for my tidyverse analyses in the first place.

I think the devil for this will be in the details of the tidyverse grammar to generate the most basic d3 "plot types", e.g. line plots, scatterplots, bar charts, faceting, and so forth, which constitute 90% of the plots one usually generates in an analysis. If the resulting tidyverse API works seamlessly for those common plots, and is flexible enough to allow customization by the user when needed, then it will have good foundations to be extended further, I reckon. The challenge then will (as usual) be to have a syntax which flows with the tidyverse operations on data and generates the right visuals out of the box effortlessly, but which is also flexible enough to make customization of visual aspects easy (or - easier than ggplot2, I often have problems with it when I want to customize stuff. I think the declarative aspect of it gets in the way).

Now, say I did some preliminary work on this consisting in: (1) Writing up a description/specification of what such a tidyverse syntax should look like (2) Implementing it as a minimal MVP with some of the basic geoms mentioned above

Would you be willing to look at it and provide your criticism? Getting the "basics" right will be most of the work.

Regards, Riccardo.

[jjallaire]

I doubt that I would have much insight on higher level plotting APIs but I'm sure lots of people would be interested in providing feedback for this work once you've got a spec and MVP. One thing to consider: if this is going to be an R-only wrapper for D3 I don't know if using r2d3 buys you much over just writing a set of htmlwidgets. e.g. you'd want to be using the advanced rendering callbacks (https://rstudio.github.io/r2d3/articles/advanced_rendering.html) which are pretty close to an htmlwidget in structure, semantics, and effort.

[riccardopinosio]

Indeed, that's a fair point! I'll see what works best, come up from an MVP, and take it from there. Thanks.

[RobertMyles]

@riccardopinosio nice idea, perhaps something like Python's Altair is what you mean?

[riccardopinosio]

@RobertMyles to a certain extent, but not completely. By giving a cursory look at your link it seems that Altair is based on Vega and the API is mostly declarative (though there seems to be some low level bindings but I'm not sure how flexible they are). In that it is then closely related to something like vegalite for R.

What I would like to have for myself would basically be a tidyverse plotting library that allows for customization of plotting elements using functional programming in a similar style as D3 callback functions or R apply/purrr family of functions. To get an idea, say I have a line plot p. I would like to be able to do something like:

p %>% transform(x_axis$labels, function(l,i){...})

Where function(l,i) is a function that takes as parameters a label element l in the label list x_axis$labels, together with the index of this element in the list i, and returns a transformed label element. It seems to me that such a "functional api" would allow for easier customization of plots without being overly complicated as in D3. For instance, let's say you want to make all the labels of the x axis for the line plot p with even index bold. Then you would do (still in pseudocode):

p %>% transform(x_axis$label, function(l,i){ if (i % 2 == 0){ return(make_bold(l)) } else { return(l) } })

When I compare what I would have to do in e.g. ggplot2 to achieve the same effect it seems to me this way of computing on graphical elements has many advantages.