implement aggregate transformer

100pah commented 3 years ago

At present the there is only a simplest aggregate transformer in echarts-simple-transform/aggregate. We should better implement a built-in aggregate transformer in echarts.

About the API

Think of whether the API of echarts-simple-transform/aggregate appropriate.

About group by

In echarts-simple-transform/aggregate only on dimension can be set in group by. Do we need to support multiple dimension group by?

About Methods

Some methods of aggregate already roughly implemented in echarts-simple-transform/aggregate:

sum
count
first
average
Q1
Q2/median
Q3
min
max

We need to make sure they are implemented correctly.

Other methods we may implement:

distinct: The count of distinct dimension values.
variance: The sample variance of dimension values.
variancep: The population variance of dimension values.
stdev: The sample standard deviation of dimension values.
stdevp: The population standard deviation of dimension values.
stderr: The standard error of dimension values.
product: The product of dimension values.
valid: The count of dimension values that are not null, undefined, '', NaN, '-', or determined by user defined validation function.
... anything else useful?

Some references: vega aggregate.

About math calculation precision

We all known that the floating precision issue (like 0.1 + 0.2). It might bring trouble in some scenarios like:

The math result is needed to be displayed in label.
The sum should equal to 100%.
Multiple by 100, 1000, ...

Some cases:

0.1 + 0.2 === 0.30000000000000004
2.3 - 0.3 === 1.9999999999999998
33643.2 - 15918.3 === 17724.899999999998
146.39 - 62.83 === 83.55999999999999
1.09 - 0.13 === 0.9600000000000001
0.1 * 0.2 === 0.020000000000000004
16.08 * 100 === 1607.9999999999998
146.39 * 100 === 14638.999999999998
9999999999999999 === 10000000000000000
-11.699999999999986 === -11.699999999999987

We should also take care the toFixed bug like:

1.005.toFixed(2) === '1.00' // rather than '1.01'
0.1.toFixed(20) === '0.10000000000000000555'

Solution:

A. Do not care about it until some users required.

B. Try to resolve it:

For arithmetic: use addSafe we already have and implement multipleSafe in appropriate place.

For comparison of two numbers, use:

const EPSILON = (function () { // Solve compatibility issues
return Number.EPSILON ? Number.EPSILON : Math.pow(2, -52);
})();
function compare(a, b){
return Math.abs(a - b) < EPSILON;
}

See: IEEE 754, http://0.30000000000000004.com/, https://github.com/camsong/blog/issues/9, https://javascript.info/number#imprecise-calculations, ... (lots of related references)

Some library:

https://github.com/nefe/number-precision

mathjs

(math.add(math.bignumber(0.1),math.bignumber(0.2)))

decimal.js

x = new Decimal(0.1)
y = 0.2
console.log(x.plus(y).toString()

bignumber.js
big.js

About big number

Do not support.

Test

Could find some test cases in some other math libraries.

maneetgoyal commented 2 years ago

Would be great to have this feature 💯

sumitnegi7 commented 2 years ago

@100pah Seems like a pretty useful feature

saikat-samanta commented 2 years ago

This feature seems really helpful. 👍

Prit-sangani commented 2 years ago

Yes, this feature is very helpful. Would be great to have this feature.

VirajBhatt9 commented 2 years ago

It is a very great feature. It would be very helpful to have this feature.

swapnil0545 commented 1 year ago

Any update on merging this?

MrZhouZh commented 10 months ago

This feature that i really want it. cannot wait

apache / echarts