holoviz-topics / neuro

HoloViz+Bokeh for Neuroscience
BSD 3-Clause "New" or "Revised" License
19 stars 5 forks source link

Add benchmark for interaction (zoom) #73

Closed ianthomas23 closed 1 year ago

ianthomas23 commented 1 year ago

This adds a benchmark for zooming an existing Bokeh figure, so records the time taken for re-rendering the figure (and any recalculation prior to that required for the re-render). It is achieved by creating a button whose callback zooms the figure so the Playwright functionality is to locate the button and click it in the same way as already performed for the latency benchmark. All of the important work occurs in JavaScript in the browser (and possibly GPU), except for some constant-ish overhead of sending the zoom request from the Python side and waiting for the render completion message to arrive on the Python side.

This is part of issue #66.

Summary of changes:

  1. There is now support for multiple Bokeh figures per benchmark. This isn't actually used yet but is a better approach that will be needed in future.
  2. Base benchmark sets the number and warmup_time of benchmark runs. This is really important here as the default asv behaviour assumes that the results of the setup stage are not affected by the actual benchmark runs whereas here that is not the case.
  3. Renaming the benchmarks from Timeseries to BokehExample.
  4. Some refactoring, pushing functionality down into the Base class so that the actual benchmark classes are as simple as possible.
  5. I have removed the n = 10_000_000 runs as this sometimes gives problems with timeouts that I have yet to solve.

On my M1 mac (no dedicated GPU) I obtain the following:

$ asv run -e
· Creating environments
· Discovering benchmarks
· Running 2 total benchmarks (1 commits * 1 environments * 2 benchmarks)
[  0.00%] · For hvneuro commit 0388952c <main>:
[  0.00%] ·· Benchmarking virtualenv-py3.11-playwright
[ 25.00%] ··· Running (bokeh_example.BokehExampleLatency.time_latency--).
[ 50.00%] ··· Running (bokeh_example.BokehExampleZoom.time_zoom--).
[ 75.00%] ··· bokeh_example.BokehExampleLatency.time_latency                                                                                                          ok
[ 75.00%] ··· ========= ============ ============
              --              output_backend     
              --------- -------------------------
                  n        canvas       webgl    
              ========= ============ ============
                 1000     84.7±5ms     90.4±5ms  
                10000     103±8ms      111±7ms   
                100000    238±4ms      240±4ms   
               1000000   1.71±0.01s   1.69±0.01s 
              ========= ============ ============

[100.00%] ··· bokeh_example.BokehExampleZoom.time_zoom                                                                                                                ok
[100.00%] ··· ========= ========== ============
              --             output_backend    
              --------- -----------------------
                  n       canvas      webgl    
              ========= ========== ============
                 1000    29.0±2ms    37.5±3ms  
                10000    30.5±1ms   33.0±0.7ms 
                100000   32.0±2ms    40.2±3ms  
               1000000   220±8ms     162±9ms   
              ========= ========== ============
droumis commented 1 year ago

noice

 asv run -e
· Creating environments...
· Discovering benchmarks.
·· Uninstalling from virtualenv-py3.11-playwright
·· Building 359a708a <main> for virtualenv-py3.11-playwright.....
·· Installing 359a708a <main> into virtualenv-py3.11-playwright....................
· Running 2 total benchmarks (1 commits * 1 environments * 2 benchmarks)
[  0.00%] · For hvneuro commit 359a708a <main>:
[  0.00%] ·· Benchmarking virtualenv-py3.11-playwright
[ 25.00%] ··· Running (bokeh_example.BokehExampleLatency.time_latency--).
[ 50.00%] ··· Running (bokeh_example.BokehExampleZoom.time_zoom--).
[ 75.00%] ··· bokeh_example.BokehExampleLatency.time_latency                                                                                                                                               ok
[ 75.00%] ··· ========= ============ ============
              --              output_backend     
              --------- -------------------------
                  n        canvas       webgl    
              ========= ============ ============
                 1000     70.1±3ms     79.2±3ms  
                10000     89.6±1ms     104±3ms   
                100000    226±4ms      230±6ms   
               1000000   1.79±0.02s   1.72±0.02s 
              ========= ============ ============

[100.00%] ··· bokeh_example.BokehExampleZoom.time_zoom                                                                                                                                                     ok
[100.00%] ··· ========= ========== ==========
              --            output_backend   
              --------- ---------------------
                  n       canvas     webgl   
              ========= ========== ==========
                 1000    52.9±3ms   54.3±4ms 
                10000    50.6±4ms   47.2±4ms 
                100000   45.0±4ms   48.3±6ms 
               1000000   214±7ms    165±3ms  
              ========= ========== ==========
droumis commented 1 year ago

Checking the variation:

asv run -b time_zoom --record-samples and then see results using something like asv show <hash> --details where the hash comes from the output of asv show

image

image