d3 / d3-shape

Graphical primitives for visualization, such as lines and areas.
https://d3js.org/d3-shape
ISC License
2.47k stars 304 forks source link

`sampling()` accessor to allow data sampling for `d3.area` and `d3.line` #197

Closed gpiroux closed 2 years ago

gpiroux commented 2 years ago

Creation of an accessor sampling to allow d3.area and d3.line sampling (skip data point based on a function), based on the accessor defined.

curran commented 2 years ago

Does this correspond to any open issue?

What problem is this solving?

Why would the sampling be done here rather than as an external data processing step?

gpiroux commented 2 years ago

@curran No open issue... We are re-developing a visualisation tool using D3 for a multi-sensors device. The idea is to display acquisition data (time series) with some analysis tools as well as dragging, zoom features ...

Depending on the sensor frequency (could be up to 4800Hz) and the zoom level, the svg.path tries to display a huge amount of data per pixel (time range around 10sec). The new d3.line.sampling(...) accessor is used in combinaison with d3.line.y(...) to aggregate data points to reduce the number of points in the svg.path dynamically (ie a few data points per pixel). We use also the area to create envelope (min, max) as an aggregation method...

Here is a screenshot of an svg created using the method described above ; 4800Hz (pt/sec), 400px for 24sec => 300 pt/px without sampling :

Screenshot 2022-05-04 at 16 03 06
curran commented 2 years ago

Got it! Thank you for explaining.

It seems quite similar to the adaptive resampling done in d3-geo.

mbostock commented 2 years ago

Is this different than filtering the data before invoking the line or area generator?

If you want to aggregate your data, I would consider looking at Observable Plot’s bin transform which provides a flexible mechanism for binning and then reducing values (e.g., to min and max or median or mean).

gpiroux commented 2 years ago

Is this different than filtering the data before invoking the line or area generator?

The idea is to bind the original data to the svg.path DOM element and use the dynamic resampling for display purpose. We need to be able to perform some data analysis on the original data set as regression, FFT ...

If you want to aggregate your data, I would consider looking at Observable Plot’s bin transform which provides a flexible mechanism for binning and then reducing values (e.g., to min and max or median or mean).

I'll have a look. Thanks !

mbostock commented 2 years ago

The idea is to bind the original data to the svg.path DOM element and use the dynamic resampling for display purpose.

You can still do that; there’s no requirement that you pass the line generator the same data that is bound to the selected element.

selection.datum(data).attr("d", line(data.filter(samplingFunction)))
gpiroux commented 2 years ago

The idea is to bind the original data to the svg.path DOM element and use the dynamic resampling for display purpose.

You can still do that; there’s no requirement that you pass the line generator the same data that is bound to the selected element.

selection.datum(data).attr("d", line(data.filter(samplingFunction)))

Yes, it is the way to go ! Thanks