search

tensorflow / lattice

Lattice methods in TensorFlow
Apache License 2.0
518 stars 94 forks source link

Lattice Kernel Tensors not Monotonic #50

Closed PoorvaRane closed 4 years ago

PoorvaRane commented 4 years ago

Referencing the issue here - https://github.com/tensorflow/lattice/issues/49

What is encoded in the lattice kernel? The lattice kernels for most of my signals look monotonic and it appears as though like tf.cumsum is not needed. However, for one set of signals, the lattice kernel looks like this:

tensor_name:  groupwise_dnn_v2/group_score/another_sample_layer/lattice_kernel
array([[0.00000],
       [0.00000],
       [0.00000],
       [0.63704],
       [0.00000],
       [0.42078],
       [0.99970],
       [1.00000],
       [0.45677],
       [0.59488],
       [1.00000],
       [1.00000],
       [0.66063],
       [0.99998],
       [1.00000],
       [1.00000],
       [0.00000],
       [0.00000],
       [0.00000],
       [0.63867],
       [0.00000],
       [0.42078],
       [0.99982],
       [1.00000],
       [0.45697],
       [0.61441],
       [1.00000],
       [1.00000],
       [0.67717],
       [0.99998],
       [1.00000],
       [1.00000],
       [0.00000],
       [0.00000],
       [0.00000],
       [0.63715],
       [0.00000],
       [0.42114],
       [1.00000],
       [1.00000],
       [0.45767],
       [0.59587],
       [1.00000],
       [1.00000],
       [1.00000],
       [1.00000],
       [1.00000],
       [1.00000],
       [0.00000],
       [0.00000],
       [0.00000],
       [0.63878],
       [0.00000],
       [0.42114],
       [1.00000],
       [1.00000],
       [0.45767],
       [0.61565],
       [1.00000],
       [1.00000],
       [1.00000],
       [1.00000],
       [1.00000],
       [1.00000]], dtype=float32)

My code for the lattice is:

tfl.layers.Lattice(
        lattice_sizes=[2]*len(sample_input),
        monotonicities=['increasing']*len(sample_input),
        output_min=0.0,
        output_max=1.0,
        name='sample_lattice'
      )(sample_calib_layer)
mmilanifard commented 4 years ago

Lattice kernel is flattened output values on each vertex (unlike pwl where the kernel encodes the deltas). The kernel you have here is monotonic. e.g. the deltas across the 3rd dim are non-negative.

kernel = tf.reshape(tf.constant(layer.kernel), [2]*6)
assert tf.reduce_min(kernel[:,:,1:,...] - kernel[:,:,:-1,...]) >= 0

The reason behind this design choice was mentioned briefly in the RFC. TL;DR is that these choices speed up training and evaluation.