search

tensorflow / lattice

Lattice methods in TensorFlow
Apache License 2.0
518 stars 94 forks source link

Is Crystals algorithm implemented in this repo? #26

Closed arrowx123 closed 4 years ago

arrowx123 commented 6 years ago

I am trying to reproduce the results shown in Experiment 1 of the Fast and Flexible Monotonic Functions with Ensembles of Lattices. The Crystals algorithm for feature selection looks very promising, is there any way that I can use it with this repo or implement it using the current infrastructure? Thank you very much for the help!

mmilanifard commented 6 years ago

Implementing crystals is a non-trivial task since tensorflow does not natively support changing the graph structure mid training, but it can be done by running one sessions with an all-pairs model (constructed using the layer API), then extracting the torsion values out of the snapshot of the trained all-pairs model, and then using that to construct the final ensemble.

I suggest instead using the embedded tiny lattices already provided in this repo. These models do not do explicit feature selection/arrangement, but rather use a linear embedding of all input features for each lattice in the ensemble. Note that this is a strict superset of the feature selection (since a linear embedding can represent a subset selection), so you might in fact get better results out of ETLs compared to crystals (at some cost to model complexity and speed).

arrowx123 commented 6 years ago

Hi Mahdi, thank you very for the detailed information! ❤️ I think another issue in terms of implementing crystals is that, for the current infrastructure, you can not set the features used for every single lattice of the ensemble.

I will definitely play with ETL models!