More broken outputs with .options

[Gordon90s]

I think related to issue #2946, it seems to me that there are some bugs that randomly show up when using .options for fine-tuning graphics. The issues this time are the following:

1) when changing options (like color) on hv_points = hv.Points((x,y)), it somehow resets .redim.range on an unrelated (?) overlay when plotting together in a layout further down. (With any attempt at re-resetting so far unsuccessful.) 2) options_dmap = {'DynamicMap': dict(height=350, width=350)} breaks everything when used on a dynamic map, though .options(height=350, width=350) does not. 3) in general, I had to split up my options all over the place to make them function correctly. I would have liked to create one large option to add at the complete end, i.e. like least_square_viz.options(options).

Here would be the whole code with comments about the issues at the specific locations:

# Visualization of the least square algorithm with slope a and offset b

import numpy as np
import holoviews as hv
hv.extension('bokeh')
import bokeh.palettes as bp # for color palettes

# set up model
np.random.seed(22)  # set seed for reproducability
n_points = 10  # number of points for the regression
errors = np.random.normal(scale = 1, size = n_points)
a = 1.0  # slope
b = -3.0  # offset
x = np.linspace(1,n_points,n_points)  # create x vector
y = a*x+b+errors  # simulated data

# define rectangle function to later draw squares
def rectangle(c=0, d=0, width=.05, height=.05):
    return np.array([(c,d), (c-width, d), (c-width, d-height), (c, d-height)])

# calculate estimates for y, error and sum of squared errors
def predictions(a_est=1, b_est=-3):
    y_est = a_est*x+b_est
    error_est = y - y_est
    sum_of_errors_est = sum(error_est**2)
    # first item returns y_est, second item returns error_est, third sum of squared errors
    return np.asarray([y_est,error_est,sum_of_errors_est])

# plot styling options
color_palette = 'PuRd'
col_1 = bp.all_palettes[color_palette][9][1]
col_2 = bp.all_palettes[color_palette][9][2]
col_3 = bp.all_palettes[color_palette][9][2]
options_curve = {'Curve': dict(color='red')}
options_points = {'Points': dict(marker='o', size=8,height=350, width=350)}
options_polygons = {'Polygons': dict(color=col_3)}
# options_dmap = {'DynamicMap': dict(height=350, width=350)}  # not working...

# create HoloViews objects
# simulated data
hv_points = hv.Points((x,y))#.options(options_points) why does that destroy my graphic?
# it destroys not even the graphic it is involved in (?)
# adding this everywhere does not help either #.redim.range(a_est = (a_est_min, a_est_max), b_est = (b_est_min, b_est_max))

# y=a_est*x+b_est line
def hv_line_est(a_est, b_est):
    hv_curve = hv.Curve((x,predictions(a_est, b_est)[0])).options(options_curve) # squared errors visualized as squares
    hv_curve = hv_curve.relabel("sum_of_squares: {}".format(np.round(predictions(a_est, b_est)[2],2)))  # add sum of squares to graphic # I would like this output not as legend but on top
    return hv_curve
def hv_squares(a_est, b_est):
    hv_polygons = hv.Polygons([{('x', 'y'): rectangle(x, y, w, w)}
             for x, y, w in np.c_[x,y,predictions(a_est, b_est)[1]]]).options(options_polygons)
    return hv_polygons
# create point of coordinate (a_est,b_est)
def hv_point_est(a_est, b_est):
    return hv.Points((a_est,b_est)).options(options_points)

# define estimate ranges and vectors for a_est and b_est
n_est = 100  # number of estimates for each a_est and b_est
a_est_min = 0.7
a_est_max = 1.3
b_est_min = -4.0
b_est_max = -2.0
a_est_vector = np.linspace(a_est_min,a_est_max,n_est)
b_est_vector = np.linspace(b_est_min,b_est_max,n_est)

# create matrix with squared error depending on a_est and b_est in the goal to draw contour lines
matrix = np.zeros((n_est,n_est))
for i in range(n_est):
    for j in range(n_est):
        matrix[i,j] = predictions(a_est_vector[j], b_est_vector[-i-1])[2]  # these are the sum of squares
bounds = (a_est_min,b_est_min,a_est_max,b_est_max)  # bounds for the coming plot
a_est_grid, b_est_grid = np.meshgrid(a_est_vector, b_est_vector)  # create input grid
hv_img = hv.Image(a_est_grid + b_est_grid, ['a estimate','b estimate'], bounds = bounds).options(cmap='PuRd',height=350, width=350)  # initialize plot; hv.Image content is added in next line
hv_img.data = matrix  # overwrite data with data we want
contour_levels=50
cmap_custom = hv.plotting.util.polylinear_gradient(['#000000', '#000000'], 20)  # color map in only black
hv_contours = hv_img * hv.operation.contours(hv_img, levels=contour_levels).options(show_legend=False, cmap=cmap_custom)

# set up dimensions for kdims for coming dynamic maps
dim_a_est = hv.Dimension('a_est', range=(a_est_min, a_est_max), default=1)
dim_b_est = hv.Dimension('b_est', range=(b_est_min, b_est_max), default=-3.0)

# dynamic maps
dmap_coords_a_b_est = hv.DynamicMap(hv_point_est, kdims=[dim_a_est,dim_b_est])
dmap_squares = hv.DynamicMap(hv_squares, kdims=[dim_a_est,dim_b_est]).options(height=350, width=350)  #options(options_dmap) # why does this not work?
dmap_line_est = hv.DynamicMap(hv_line_est, kdims=[dim_a_est,dim_b_est])
ls_layout = (dmap_squares * dmap_line_est * hv_points).redim.range(x=(0,11.5), y=(-2.5,7.8))

# generate layouts
contour_layout = hv_contours * dmap_coords_a_b_est
least_square_viz = (ls_layout + contour_layout)
least_square_viz 

Is there in general something that allows to set settings like global magics for pure python? I guess I maybe should use get_ipython().magic for that? I had the impression that magics worked better.

Finally, related to #2948, I would like to have on the final layout least_square_viz, the following show up on the top left corner: .relabel("sum_of_squares: {}".format(np.round(predictions(a_est, b_est)[2],2))). (Instead of currently as a legend + a_est: ..., b_est: ....)

[Gordon90s]

I managed to solve a few problems since.

About 1): This seems to be a minor bug after all. Somehow, the option 'Layout': dict(normalize=True), which is the default value, only became active once I changed the options for 'Points'. I.e. by adding 'Layout': dict(normalize=False), everything worked as I hoped again. About 2): With dmap_squares = hv.DynamicMap(hv_squares, kdims=[dim_a_est,dim_b_est]).options(height=350, width=350), I mistakenly thought that the object 'DynamicMap' had the options height and width, which is not the case. That is what explains why options_dmap = {'DynamicMap': dict(height=350, width=350)} broke. Though .options(height=350, width=350) seems to be somehow transmitted to a lower level, which reinforced my mistaken belief in the first place. About 3): With 2) out of the way, I was able to achieve 3).

About .relabel("sum_of_squares: {}".format(np.round(predictions(a_est, b_est)[2],2))): I tried a few things, even just to control the location of the generated legend without success (I would have liked it to be on the left). At least I deleted the title with title_format='' and I'll leave the visualization like it is now. This would be the latest plot version:

# Visualization of the least square algorithm with slope a and offset b

import numpy as np
import holoviews as hv
hv.extension('bokeh')
import bokeh.palettes as bp # for color palettes

# set up model
np.random.seed(22)  # set seed for reproducability
n_points = 10  # number of points for the regression
errors = np.random.normal(scale = 1, size = n_points)
a = 1.0  # slope
b = -3.0  # offset
x = np.linspace(1,n_points,n_points)  # create x vector
y = a*x+b+errors  # simulated data

# define rectangle function to later draw squares
def rectangle(c=0, d=0, width=.05, height=.05):
    return np.array([(c,d), (c-width, d), (c-width, d-height), (c, d-height)])

# calculate estimates for y, error and sum of squared errors
def predictions(a_est=1, b_est=-3):
    y_est = a_est*x+b_est
    error_est = y - y_est
    sum_of_errors_est = sum(error_est**2)
    # first item returns y_est, second item returns error_est, third sum of squared errors
    return np.asarray([y_est,error_est,sum_of_errors_est])

# create HoloViews objects
## simulated data
hv_points = hv.Points((x,y))

## y=a_est*x+b_est line
def hv_line_est(a_est, b_est):
    hv_curve = hv.Curve((x,predictions(a_est, b_est)[0])) 
    hv_curve = hv_curve.relabel("sum_of_squares: {}".format(np.round(predictions(a_est, b_est)[2],2)))  # add sum of squares to graphic # I would like this output not as legend but on top
    return hv_curve

## squared errors visualized as squares
def hv_squares(a_est, b_est):
    hv_polygons = hv.Polygons([{('x', 'y'): rectangle(x, y, w, w)}
             for x, y, w in np.c_[x,y,predictions(a_est, b_est)[1]]])
    return hv_polygons

## create point of coordinate (a_est,b_est)
def hv_point_est(a_est, b_est):
    return hv.Points((a_est,b_est)).options(marker='o', size=10, color='red')

# define estimate ranges and vectors for a_est and b_est
n_est = 100  # number of estimates for each a_est and b_est
a_est_min = 0.7
a_est_max = 1.3
b_est_min = -4.0
b_est_max = -2.0
a_est_vector = np.linspace(a_est_min,a_est_max,n_est)
b_est_vector = np.linspace(b_est_min,b_est_max,n_est)

# create matrix with squared error depending on a_est and b_est in the goal to draw contour lines
matrix = np.zeros((n_est,n_est))
for i in range(n_est):
    for j in range(n_est):
        matrix[i,j] = predictions(a_est_vector[j], b_est_vector[-i-1])[2]  # these are the sum of squares
bounds = (a_est_min,b_est_min,a_est_max,b_est_max)  # bounds for the coming plot
a_est_grid, b_est_grid = np.meshgrid(a_est_vector, b_est_vector)  # create input grid
hv_img = hv.Image(a_est_grid + b_est_grid, ['a estimate','b estimate'], bounds = bounds)  # initialize plot; hv.Image content is added in next line
hv_img.data = matrix  # overwrite data with data we want
contour_levels=50
hv_contours = hv_img * hv.operation.contours(hv_img, levels=contour_levels)

# set up dimensions for kdims for coming dynamic maps
dim_a_est = hv.Dimension('a_est', range=(a_est_min, a_est_max), default=1)
dim_b_est = hv.Dimension('b_est', range=(b_est_min, b_est_max), default=-3.0)

# dynamic maps
dmap_coords_a_b_est = hv.DynamicMap(hv_point_est, kdims=[dim_a_est,dim_b_est])
dmap_squares = hv.DynamicMap(hv_squares, kdims=[dim_a_est,dim_b_est])
dmap_line_est = hv.DynamicMap(hv_line_est, kdims=[dim_a_est,dim_b_est])
ls_layout = (dmap_squares * dmap_line_est * hv_points.options(marker='o', size=5)).redim.range(x=(-0.5,12.5), y=(-3,10))

# plot styling options
color_palette = 'PuRd'
col_1 = bp.all_palettes[color_palette][9][1]
col_2 = bp.all_palettes[color_palette][9][2]
col_3 = bp.all_palettes[color_palette][9][2]
cmap_custom = hv.plotting.util.polylinear_gradient(['#000000', '#000000'], 2)  # color map in only black

options = {'Curve': dict(color='red', height=350, width=350),
           'Polygons': dict(color=col_3),
           'Image': dict(cmap='PuRd', height=350, width=350),
           'Contours': dict(show_legend=False, cmap=cmap_custom),
           'Layout': dict(shared_axes=False, normalize=False, title_format='')
          }                                

# generate layouts
contour_layout = hv_contours * dmap_coords_a_b_est     
least_square_viz = (ls_layout + contour_layout)
least_square_viz = least_square_viz.options(options)
least_square_viz

[philippjfr]

Thanks for getting back to this, that's a great demo and we'd love to see it added it to our examples. It also makes a good example to optimize the internals a bit. One fun change is to replace the kdims with a stream:

stream = hv.streams.PointerXY(x=1, y=-3, rename={'x': 'a_est', 'y': 'b_est'})

# dynamic maps
dmap_coords_a_b_est = hv.DynamicMap(hv_point_est, streams=[stream])
dmap_squares = hv.DynamicMap(hv_squares, streams=[stream])
dmap_line_est = hv.DynamicMap(hv_line_est, streams=[stream])

But it would be even better if it was a bit faster, for now a Tap stream seems best:

stream = hv.streams.Tap(x=1, y=-3, rename={'x': 'a_est', 'y': 'b_est'})

[Gordon90s]

Thanks for getting back to this, that's a great demo and we'd love to see it added it to our examples. Sure go for it. Glad you like it!

The stream is indeed a nice!

[thoth291]

I thought that the issues I experience are belong here: https://anaconda.org/thoth/debugging-dynamicmaps/notebook Quickly browsing through notebook would better illustrate multiple issues with disappearing ranges and frozen plots

It would be great if someone would confirm that I'm not hallucinating. @philippjfr , do you have any ideas?