si-you
commented
6 years ago Hi there!
Q1) API docs only includes that are public under tfl, and monotonic linear embedding is not exposed as a public symbol under tfl.
But the implementation can be found in here https://github.com/tensorflow/lattice/blob/master/tensorflow_lattice/python/lib/monotone_linear_layers.py
You can access this function using from tfl.python.lib.monotone_linear_layers import monotone_linear_layer from tfl.python.lib.monotone_linear_layers import split_monotone_linear_layer
Docstring contains a documentation so please read it how to use this :).
Q2) Section 3 in the paper provides an extensive explanation why we need the linear embedding. The number of inputs to a multi-d single lattice can't be larger than 16, because the number of parameter is O(2 ** 16). So ensemble of lattices is proposed in "Fast and flexible monotonic functions with ensembles of lattices", NIPS 2016 (see the paper for details how this ensemble overcomes this dimensional limitation). NIPS 2016 proposed a heuristic to choose ensemble structure (which inputs go into which lattice), but that requires pre-processing to figure out the ensemble structure. Linear embedding layer does not require this pre-processing, and in terms of a function class, it is a superset of ensemble of lattices, because for any given ensemble structure, we can find the corresponding embedding matrix whose elements are either 0 or 1. So instead of pre-specifying this embedding matrix's element 0 or 1, "Deep Lattice Networks and Partial Monotonic Functions", NIPS 2017 (to be published), proposes to jointly train this linear embedding.
wenruij
Q1: There are three kinds of layers: linear embeddings, calibrators, and ensembles of lattices mentioned in the paper, Is there an implement for linear embeddings layer? I checked the API docs, seems no one found?
Q2: Furthermore, any hint for the purpose of the linear embeddings layer?