crossfilter / reductio

Reductio: Crossfilter grouping
Other
250 stars 42 forks source link

Add downsampling feature #21

Closed nordfjord closed 9 years ago

nordfjord commented 9 years ago

This pull request adds two methods to reducers

  1. downsample
  2. threshold

    Downsample

is a value accessor for bucket creation

example:

reducer = reductio()
  .sum('foo')
  .downsample('sum');

Threshold

The length you wish to get from group.all()

example:

reductio()
  .sum('foo')
  .downsample('sum')
  .threshold(2)(group);

group.all() // returns array with length 2

note that if the reducers threshold changes at any time the change will be reflected in the group.all method

this is especially useful if you're drawing charts with the output and want to draw as little data as possible (say no more than the pixel width of the chart).

demo/

If you look in the demo directory there's a little demo utilizing dc.js for charting and a slider to change the threshold

Let me know if anything can go better in this PR, and I'll oblige happily.

nordfjord commented 9 years ago

My original thought train was that this belonged in the charting libraries themselves, but then I thought that it might be nice to have it in a crossfilter grouping all by itself so that all charting libraries might use it.

esjewett commented 9 years ago

This looks good, but 2 requests:

  1. Let's resolve where this lives on the API first (see discussion in #22).
  2. Dealing with the largest-triangle-three-buckets.js file is a bit complex. My first suggestion would be: can we get this algorithm published as a separate npm and bower module, then use it as a runtime dependency? That would probably be cleanest. If not, then we should probably put it in a 'lib' folder or something.

Regardless, I need to create NOTICE and CONTRIBUTOR files to explicitly list copyright/license notices for included stuff as well as the contributors who own copyright on specific files in Reductio (the information currently lives in pull requests and issues).

nordfjord commented 9 years ago

Hadn't thought about copyright issues surrounding this. I'll try to find a better way for this to exist in reductio.

esjewett commented 9 years ago

It's not so much that there are major issues as that I haven't yet set up the project to deal with the well-known issues around incorporating 3rd-party-copyright code. More on my end than anything else. Will continue this discussion over on the other pull request.