Client side histogramming in bokeh interface - unbinned and binned data

[miranov25]

Proposal: syntax for unbinned data histogramming on client

Standard syntax example: https://github.com/miranov25/RootInteractive/blob/master/RootInteractive/InteractiveDrawing/bokeh/test_bokehDrawSA.py

figureArray = [
#   ['A'], ['C-A'], {"color": "red", "size": 7, "colorZvar":"C", "filter": "A<0.5"}],
    [['A'], ['A*A-C*C'], {"color": "red", "size": 2, "colorZvar": "A", "varZ": "C", "errY": "errY", "errX":"0.01"}],
    [['A'], ['C+A', 'C-A', 'A/A']],
...
    [['D'], ['D*10'], {"size": 10, "errY": "errY"}],
]

->

figureArray = [
   ['A'], ['histo'], {"color": "red", "size": 7,nBins=10, range=() ],
   ['A','B'], ['histo'], {"color": "red", "size": 7, nBins=10 , range=()],
   ['A','B'], ['histo2D'], {"color": "red", "size": 7, nBins=10 , range=()],
]

[miranov25]

Histogramming on server, slicing on client

[miranov25]

Unbined histogramming on client:

figureArray = [
   ['A'], ['histo(A/B-C)'], {"color": "red", "size": 7,nBins=10, range=() ],
   ['A','B'], ['histo'], {"color": "red", "size": 7, nBins=10 , range=()],
   ['A','B'], ['histo2D'], {"color": "red", "size": 7, nBins=10 , range=()],
]

• histo(A/B)-1

• histo(f(A,b))

• histoA/histoB-1
• f(histoA,histoB)

[miranov25]

Summary of pull requests for the 1 dimensional histograms

• 92

• 93

• 94

Benchmark - 2 10^6 points with weights - solutions improvements as reported by Marian in pull requests above

• Values in table are rough as no standard benchmark on client exist yet
• New bottleneck in filtering
  • Filtering took 1007.57999998983 milliseconds.
• Histogramming of the same data for small subset selection is significantly faster"
• Selecting e.g 10 % of the range time is reduced by corresponding factor

  Filtering took 166.22499999403954 milliseconds.
  VM36:130 Histogramming took 47.06500000599772 milliseconds.

Solution	speedup factor	commit	time per 210^6 bins
Starting point			5 s
Caching bin calculation	~2	https://github.com/miranov25/RootInteractive/pull/93/commits/40b1f423c73363dca8b7971060b862b3342c5f75	~??? s
Directo function call instead of callback	~3	https://github.com/miranov25/RootInteractive/pull/93/commits/379116962d0c8d555495d67d73d90fad6fb52ab6	Histogramming took 1073.6 ms
???	~2	https://github.com/miranov25/RootInteractive/pull/93/commits/90ad2a6678af4e2b6f82958b09d3e9fa9a300e98	Histogramming took 730.9 milliseconds.
NEW using build in	~3	https://github.com/miranov25/RootInteractive/pull/94/commits/969368b92dbd3b744d84c58e02e233b34d362c28	Histogramming took 323.8 milliseconds.

[miranov25]

2d histogram and histogram of functions and functions of histograms an categories histogramming - user interface

• to follow numpy naming convention

Histogram - parameters - in Dictionary {}

• name:
• title:
• type:
  • histogram
  • profileX, profileY
  • quantileX/medianX, quantileY/medianY(see vaex option)
    • inside bin constant approximation assumed
• visualization type
  • 2D - surface
  • stack
  • individual - colz
• variables: list
  • 1D, 2D
  • custom function on client (less data to transfer)
• weights
• bins:
• nbins

Function on top of histograms:

• arbitrary JS function using names of histograms defined in section above

[miranov25]

histogramArray = [
    {"name":"histo0", "title": "histo0", type="histo", variables=["X","Y"], "weight":"C" },
    {"name":"histo1", "title": "histo1", type="histo", variables=["X","Y"], "weight":"C" },
]

to be used in figure array using names.

[miranov25]

Histogram drawing options to be defined:

• Examples from other packages :

  • Root:
  • https://root.cern.ch/root/htmldoc/guides/users-guide/Histograms.html#drawing-histograms

• Check also python histogram packages

[miranov25]

Automatic resize of histograms on client

In case histogram ranges not specified - automatic ranges should be recalculated, or we should support auto mode