Convert everything to tensors before passing to tfp

[dfalbel]

WIP

• [ ] Figure out what to do with LinearOperators
• [ ] Fix tests. Most failing tests are due to other tf ops that returns unexpected float64 values.

this may also not be the right approach.

[skeydan]

I just got a warning from TFP like this:

WARNING:tensorflow:Layer conv1d_flipout is casting an input tensor from dtype float64 to the layer's dtype of float32, which is new behavior in TensorFlow 2.  The layer has dtype float32 because it's dtype defaults to floatx.

If you intended to run this layer in float32, you can safely ignore this warning. If in doubt, this warning is likely only an issue if you are porting a TensorFlow 1.X model to TensorFlow 2.

To change all layers to have dtype float64 by default, call `tf.keras.backend.set_floatx('float64')`. To change just this layer, pass dtype='float64' to the layer constructor. If you are the author of this layer, you can disable autocasting by passing autocast=False to the base Layer constructor.

I wonder if now is the time to try something like this again, but not using

tf.convert_to_tensor

(because

dtype: Optional element type for the returned tensor. If missing, the type is inferred from the type of value.

) but

backend.floatx()

instead?

What do you think @dfalbel ?