hidden layers doesn't work with tensorflow 2.1

[Thomasillo]

Versions: python 3.6 tensorflow 2.1.0 talos 0.6.3

import tensorflow as tf
import talos
from talos.model import hidden_layers
model = tf.keras.Sequential()

hidden_layers(model, {'first_neuron': 3, 
                                'shapes': 'brick', 
                                'hidden_layers': 2, 
                                'dropout': 0, 
                                'activation': 'relu'}, 1)

Results in:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-4-ca60d86b3018> in <module>
      8                                 'hidden_layers': 2,
      9                                 'dropout': 0,
---> 10                                 'activation': 'relu'}, 1)

~/.local/miniconda3/envs/mantik_dev/lib/python3.7/site-packages/talos/model/hidden_layers.py in hidden_layers(model, params, last_neuron)
     76                         activity_regularizer=activity_regularizer,
     77                         kernel_constraint=kernel_constraint,
---> 78                         bias_constraint=bias_constraint))
     79 
     80         model.add(Dropout(params['dropout']))

~/.local/miniconda3/envs/mantik_dev/lib/python3.7/site-packages/tensorflow_core/python/training/tracking/base.py in _method_wrapper(self, *args, **kwargs)
    455     self._self_setattr_tracking = False  # pylint: disable=protected-access
    456     try:
--> 457       result = method(self, *args, **kwargs)
    458     finally:
    459       self._self_setattr_tracking = previous_value  # pylint: disable=protected-access

~/.local/miniconda3/envs/mantik_dev/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/sequential.py in add(self, layer)
    159       raise TypeError('The added layer must be '
    160                       'an instance of class Layer. '
--> 161                       'Found: ' + str(layer))
    162 
    163     tf_utils.assert_no_legacy_layers([layer])

TypeError: The added layer must be an instance of class Layer. Found: <keras.layers.core.Dense object at 0x10810e450>

Apart from that, the parameter 'dropout' and 'activation' are REQUIRED despite not being said so in the documentation.

[mikkokotila]

Better to update to latest (0.6.4).

But you can't use hidden_layers like that.

1) Make sure follow this example for the overall structure of the input model

2) See here for an indication of how to use.

Closing here. Feel free to open new issue if anything.

[Thomasillo]

Yeah, the example was flawed. But the bug is still there. I corrected the example. Try it out and reopen if applicable.

[Thomasillo]

Moreover:

$ pip install  talos==0.6.4
Looking in indexes: https://pypi.org/simple
ERROR: Could not find a version that satisfies the requirement talos==0.6.4 (from versions: 0.1.8, 0.1.9, 0.4.3, 0.4.5, 0.4.6, 0.4.7, 0.4.8, 0.4.9, 0.5.0, 0.6.3)
ERROR: No matching distribution found for talos==0.6.4

[mikkokotila]

Your input should look ~literally~ structurally like this hidden_layers(model, params, 1). The params has to be same as the variable that is used as input for the input model. Also, the example you have provided would still not work. If you want to provide a meaningful example, then that would be a working keras model in the form of the input model as per the Talos docs.

[mikkokotila]

Ah, my bad regarding 0.6.4 ... I had not released it. Moving CI from Travis to Github Actions, which came in the way of releasing it. Will try to do it in the next few days.

[Thomasillo]

This is one to one copied from your examples and it reproduces the bug.

Apparently, the reason is the import tensorflow.keras.models instead of keras.models.

I figure talos is only intended to be used with keras not with tensorflow.keras?

import talos

from talos.model import hidden_layers
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense

x, y = talos.templates.datasets.iris()

def iris_model(x_train, y_train, x_val, y_val, params):

    model = Sequential()
    model.add(Dense(32, input_dim=4, activation=params['activation']))
    hidden_layers(model, params, 1)
    model.add(Dense(3, activation='softmax'))

    model.compile(optimizer=params['optimizer'], loss=params['losses'])

    out = model.fit(x_train, y_train,
                     batch_size=params['batch_size'],
                     epochs=params['epochs'],
                     validation_data=[x_val, y_val],
                     verbose=0)

    return out, model

p = {'activation':['relu', 'elu'],
     'optimizer': ['Nadam', 'Adam'],
     'losses': ['logcosh'],
     'shapes': ['brick'],          # <<< required
     'first_neuron': [32, 64],     # <<< required
     'hidden_layers':[0, 1, 2],    # <<< required
     'dropout': [.2, .3],          # <<< required
     'batch_size': [20,30,40],
     'epochs': [200]}

scan_object = talos.Scan(x, y, experiment_name="foo", model=iris_model, params=p, fraction_limit=0.1)

[mikkokotila]

Have a look at #451 for more information regarding Tensorflow >=2.0 support. The very short version is that it's definitely the focus going forward. Legacy keras will not have bug fix support from may 1st.