search

plotly / plotly.js

Open-source JavaScript charting library behind Plotly and Dash
https://plotly.com/javascript/
MIT License
16.92k stars 1.85k forks source link

Bumping d3 #424

Open etpinard opened 8 years ago

etpinard commented 8 years ago

A discussion on what modules we need in d3 v4.0.0.

monfera commented 8 years ago

@etpinard these are probably well known to all of us, so it's a 'just in case':

n-riesco commented 8 years ago

@monfera Thx for the links!

monfera commented 8 years ago

Ann. by Mike Bostock: D3 4.0 is now in master and a release is coming up in a month or two.

rnons commented 8 years ago

D3 4.0 released

rreusser commented 8 years ago

Observations from an aborted attempt:

Beyond that, it's not like the logic is all that different. It's a lot of bouncing from one error to the next and translating the code. The result that I got was able to display a grid with dragging and some hoverlabels, so it's viable, just would take more work than seems appropriate in the short term.

monfera commented 8 years ago

@rreusser nice summary! Yes, lots of relatively superficial upgrade work even with smaller codebases. Various incompatible changes, e.g. the collapsing of the namespaces, various renamings or both (e.g. d3.layout.force -> d3.forceSimulation), or what things return (e.g. some palettes were scales but now they're arrays) or what methods need to be used.

Besides the d3.round thing, there are subtle changes, such as d3.range no longer attempting to control for numerical inaccuracy (I saw it from Irene Ros, great presentation).

I wonder if the selection[0][0] thing can be replaced with .node(). In any case the array access was never part of the API so it's a good idea to avoid it.

Regarding the promise of smaller components, maybe this new tool can help customize what's included but I haven't yet tried. The reason it might work is that there are numerous modules not used in the codebase. In any case it's an almost negligibly small fraction of the plotly.js codebase.

Based on the current way we use D3 I wouldn't expect significant user-visible improvements from a switch. It can be useful for other reasons, I suppose the community (programmers, bug fixes) move to 4.0 over time. Or if there is a big modularization attempt in plotly.js itself, e.g. now lightweight plots as in journals aren't feasible due to the load and initialization time. Some other things also came up earlier, e.g. our use of the palettes (there's duplication between plotly.js and the gl-vis packages. The force layout in 4.0 is deterministic and way faster than currently, see my three recent blocks or these two gifs from Mike Bostock.

All in all D3 4.0 doesn't seem to address a current technical problem but makes some things easier in the future. It's a tough call because most folks can leave old projects in 3.* and use 4.* on new ones but plotly.js is not a dataviz project.

rreusser commented 8 years ago

Haha, but array access is soo easy! Come to think of it, I'm not sure it's actually used in the code. That might just be what I use to debug in the inspector.

rreusser commented 8 years ago

Yeah, the other change that maybe had consequences was transitions. The current animation PR falls back to instantaneous data updates when you transition the scale and the data at the same time. The reason seems to be a race condition in which the running transitions (points moving around) don't play well with transform changes (the whole layer moving around). It's difficult to debug because of the way v3 makes it relatively difficult to group transitions (i.e. transition.select(); and specifically, difficult at least within the plotly.js codebase). v4 lets you group more precisely by simply passing a transition to a new transition.

It's ripe for debugging and refactoring, but the current priority is consistency and correctness, then handling more complicated interactions and corner cases. I think v3 will get us there. 👍

monfera commented 8 years ago

Good point. Transitions can be a huge thing to upgrade properly (the new is so much better, taking advantage of it may need full rethinking of the transitions). Since you wrote the animations you are the one who knows about the expected transition upgrade friction.

etpinard commented 7 years ago

Referencing @rreusser 's https://github.com/plotly/plotly.js/pull/946#discussion_r82048108 about our the very common .data([0]) pattern.

rreusser commented 7 years ago

Makes it an extra good reason to factor it out so it's only one fix.

agnivade commented 7 years ago

Hi there, this looks like a great project ! What is the status of upgrading to d3 v4 ? Has this been put to the backburner for now ?

monfera commented 7 years ago

@agnivade thanks! Not quite an answer, just wanted to mention that we have a little bit of use of a couple of D3 4.0 modules where it had a significant performance benefit - the use of the force layout, and its dependencies eg. quadtree in Sankey are 4.0. Force is being used to settle the diagram when moving around nodes: https://codepen.io/monfera/full/VbKrXM/

agnivade commented 7 years ago

I see. Thanks, makes sense to me.

etpinard commented 6 years ago

FYI d3 v5 should make its official debut soon: https://github.com/d3/d3/releases/tag/v5.0.0

mojoaxel commented 6 years ago

Well, it's 2018 and d3 is in v5 by now... Is plotly planning to upgrade at some point?

namoshizun commented 6 years ago

Any news on this matter? Is this D3 version bumping still considered? Have been recently using the R plotly but occasionly had to access the D3 interface. It is a bit cumbersome to use the old D3 v3..

fsmeier commented 6 years ago

Is this topic up2date? - I would propose the following: instead of replacing all the d3-usages in the code I would create a intermediate-d3-wrapper which forwards functions used in plotly to the related functions in d3. In the wrapper itself we could make a version-check of d3 and depending on the version use the right d3-function. Also with this plotly could maybe support different versions and an upgrade to later versions of d3 are maybe easier.

Does this idea/approach make sense somehow?

nite commented 5 years ago

Hi - is this still active?

etpinard commented 5 years ago

Pasting https://github.com/plotly/plotly.js/issues/3052#issuecomment-424744313

For completeness, the items d3v3 is missing that v5 has are:

%Q - milliseconds since UNIX epoch.

%s - seconds since UNIX epoch.

%u - Monday-based (ISO 8601) weekday as a decimal number [1,7].

%V - ISO 8601 week of the year as a decimal number [01, 53].

%f - microseconds as a decimal number [000000, 999999]. Frustratingly, d3 did something different than we did with %f, which we added (before it was part of d3) to support more digits and increased flexibility over %L. They added it as a fixed 6 digits of precision, whereas we support any precision up to 6 digits (using %{n}f) and we truncate trailing zeros. The fixed precision of only 3 (%L) or 6 digits is, IMO, a significant limitation (though I can see wanting to keep trailing zeros, we could easily add that) so I guess we'll be keeping this particular difference from D3 long-term...

Perhaps not worth it until we upgrade our d3, but it would be possible for us to add Q,s,u,V similarly to how we added f.

nite commented 4 years ago

d3-format v4+ has new formatting features ...

eg.

d3.format("(0.2s")(-1500);
(2k)
bishonen commented 4 years ago

if there's no real plans to update to the newer version of d3, is there some recommendation for users to still end up with a ISO8601 weeknum (%V)?

kristapskristaps commented 4 years ago

if there's no real plans to update to the newer version of d3, is there some recommendation for users to still end up with a ISO8601 weeknum (%V)?

Is there any plan to implement this?

sebhmg commented 4 years ago

Also especially usefully in the new d3-format:

The ~ option trims insignificant trailing zeros

Finally, In order to have nice tick positioning and formatting while zooming, I had to use newer d3 (5.15) on the side. I imagine Plotly would just take care of it, once upgraded to a newer version of d3.

Meanwhile, here is my code. Upon ploty_relayout event:

        // use d3 to compute ticks (a newer version than the one used by Plotly)
        const xNiceScale = d3.scaleLinear().domain(xaxisRange).nice();
        const xTickValues = xNiceScale.ticks();
        const xTickFormatter = xNiceScale.tickFormat(xTickValues.length);
        const xTickLabels = xTickValues.map(xTickFormatter);

        const yNiceScale = d3.scaleLinear().domain(yaxisRange).nice();
        const yTickValues = yNiceScale.ticks();
        const yTickFormatter = yNiceScale.tickFormat(yTickValues.length);
        const yTickLabels = yTickValues.map(yTickFormatter);

        const layout = {
            "xaxis.tickmode": "array",
            "xaxis.tickvals": xTickValues,
            "xaxis.ticktext": xTickLabels,
            "yaxis.tickmode": "array",
            "yaxis.tickvals": yTickValues,
            "yaxis.ticktext": yTickLabels
        };
archmoj commented 4 years ago

if there's no real plans to update to the newer version of d3, is there some recommendation for users to still end up with a ISO8601 weeknum (%V)?

This one is addressed in #5026 and would be available in plotly.js v1.55.0.

archmoj commented 4 years ago

A table of d3-v3 methods applied in current plotly.js. The +/- sign indicates that an equivalent v5 sub-module found and could possibly be replaced.

- d3.behavior
+ d3.dispatch
+ d3.ease
- d3.event
+ d3.extent
+ d3.format
+ d3.geo
+ d3.hsl
+ d3.interpolate
+ d3.json
- d3.locale
+ d3.max
+ d3.merge
+ d3.min
+ d3.mouse
+ d3.nest
+ d3.range
- d3.rebind
+ d3.rgb
- d3.round
- d3.scale
+ d3.select
+ d3.sum
+ d3.svg
- d3.time
+ d3.transition
+ d3.zip
archmoj commented 4 years ago

:tada: and now d3-v6 is out: https://observablehq.com/d/f91cccf0cad5e9cb

jackparmer commented 4 years ago

This issue has been tagged with NEEDS SPON$OR

A community PR for this feature would certainly be welcome, but our experience is deeper features like this are difficult to complete without the Plotly maintainers leading the effort.

Sponsorship range: $70k-$80k

What Sponsorship includes:

Please include the link to this issue when contacting us to discuss.

jacksongoode commented 1 year ago

Just to be clear... Plotly is based on a version (~3.x) of d3 from 2016... The current version of d3 is now 7.4.5. This seems like a major concern given that it's the foundational package for graphing.

gvwilson commented 1 month ago

@emilykl before you move on, could you please create a confluence page with your findings?