Metaclass conflict with Keras 2.3.0

Using HyperparameterHunter with the recently released Keras version 2.3.0 raises metaclass conflict error, breaking Keras compatibility
TEMPORARY WORKAROUND: Use Keras 2.2.5

Error Traceback

From executing examples/keras_examples/experiment_example.py

Traceback (most recent call last):
File "keras_examples/experiment_example.py", line 1, in <module>
from hyperparameter_hunter import Environment, CVExperiment
File "/Users/Hunter/hyperparameter_hunter/hyperparameter_hunter/__init__.py", line 29, in <module>
from .environment import Environment
File "/Users/Hunter/hyperparameter_hunter/hyperparameter_hunter/environment.py", line 27, in <module>
from hyperparameter_hunter.keys.makers import CrossExperimentKeyMaker
File "/Users/Hunter/hyperparameter_hunter/hyperparameter_hunter/keys/makers.py", line 28, in <module>
from hyperparameter_hunter.library_helpers.keras_optimization_helper import initialize_dummy_model
File "/Users/Hunter/hyperparameter_hunter/hyperparameter_hunter/library_helpers/keras_optimization_helper.py", line 52, in <module>
from keras.callbacks import Callback as base_keras_callback
File "/Users/Hunter/hyperparameter_hunter/env/lib/python3.7/site-packages/keras/__init__.py", line 3, in <module>
from . import utils
File "/Users/Hunter/hyperparameter_hunter/env/lib/python3.7/site-packages/keras/utils/__init__.py", line 26, in <module>
from .vis_utils import model_to_dot
File "/Users/Hunter/hyperparameter_hunter/env/lib/python3.7/site-packages/keras/utils/vis_utils.py", line 7, in <module>
from ..models import Model
File "/Users/Hunter/hyperparameter_hunter/env/lib/python3.7/site-packages/keras/models.py", line 10, in <module>
from .engine.input_layer import Input
File "/Users/Hunter/hyperparameter_hunter/env/lib/python3.7/site-packages/keras/engine/__init__.py", line 8, in <module>
from .training import Model
File "/Users/Hunter/hyperparameter_hunter/env/lib/python3.7/site-packages/keras/engine/training.py", line 14, in <module>
from . import training_utils
File "/Users/Hunter/hyperparameter_hunter/env/lib/python3.7/site-packages/keras/engine/training_utils.py", line 17, in <module>
from .. import metrics as metrics_module
File "/Users/Hunter/hyperparameter_hunter/env/lib/python3.7/site-packages/keras/metrics.py", line 34, in <module>
class Metric(Layer):
File "/Users/Hunter/hyperparameter_hunter/env/lib/python3.7/site-packages/six.py", line 849, in wrapper
return metaclass(cls.__name__, cls.__bases__, orig_vars)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/abc.py", line 126, in __new__
cls = super().__new__(mcls, name, bases, namespace, **kwargs)
TypeError: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases

Bug Hunt Notes

Updating tracers.ArgumentTracer to descend from ABCMeta does resolve the metaclass conflict, but leads to different errors regarding the usage of __deepcopy__ in TensorFlow:
- NotImplementedError: __deepcopy__() is only available when eager execution is enabled.
Further changes to the Keras API cause some unrelated tests to fail
Noteworthy changes in Keras 2.3.0 that will require HH fixes:
- Introduced class-based losses
- Introduced class-based metrics
- Renamed lr to learning_rate for all optimizers
- Deprecated decay argument for all optimizers
- Changed how metric names are reported to use the exact string given by the user ("acc" vs. "accuracy")

Subclassing ArgumentTracer(abc.ABCMeta) in hyperparameter_hunter.tracers.py, then applying the following fixes yields the error shown toward bottom -- also shown output for print(self.name) inserted right above the breaking code.

# keras_helper.py
# AttributeError: 'LossFunctionWrapper' object has no attribute '__name__'
try:
    compile_params["loss_function_names"] = [_.__name__ for _ in compile_params["loss_functions"]]
except:
    compile_params["loss_function_names"] = [_.name for _ in compile_params["loss_functions"]]

# AttributeError: 'Sequential' object has no attribute 'weighted_metrics'
try:
    compile_params["weighted_metrics"] = get_keras_attr(model, "weighted_metrics")
except:
    print('WARNING: could not fetch "weighted_metrics" attribute; skipping...')

Full error trace:

total:0 # print(self.name) outputs
count:0
UnreadVariable

Traceback (most recent call last):

  File "<ipython-input-1-1226ec5313a7>", line 56, in <module>
    execute()
  File "<ipython-input-1-1226ec5313a7>", line 49, in execute
    shuffle=True,
  File "C:\hyperparameter_hunter\hyperparameter_hunter\experiment_core.py", line 165, in __call__
    return super().__call__(*args, **kwargs)
  File "C:\hyperparameter_hunter\hyperparameter_hunter\experiments.py", line 749, in __init__
    target_metric=target_metric,
  File "C:\hyperparameter_hunter\hyperparameter_hunter\experiments.py", line 595, in __init__
    target_metric=target_metric,
  File "C:\hyperparameter_hunter\hyperparameter_hunter\experiments.py", line 302, in __init__
    self.preparation_workflow()
  File "C:\hyperparameter_hunter\hyperparameter_hunter\experiments.py", line 354, in preparation_workflow
    self._generate_hyperparameter_key()
  File "C:\hyperparameter_hunter\hyperparameter_hunter\experiments.py", line 450, in _generate_hyperparameter_key
    self.hyperparameter_key = HyperparameterKeyMaker(parameters, self.cross_experiment_key)
  File "C:\hyperparameter_hunter\hyperparameter_hunter\keys\makers.py", line 367, in __init__
    KeyMaker.__init__(self, parameters, **kwargs)
  File "C:\hyperparameter_hunter\hyperparameter_hunter\keys\makers.py", line 90, in __init__
    self.parameters = deepcopy(parameters)
  File "D:\Anaconda\envs\test_hh\lib\copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "D:\Anaconda\envs\test_hh\lib\copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "D:\Anaconda\envs\test_hh\lib\copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "D:\Anaconda\envs\test_hh\lib\copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "D:\Anaconda\envs\test_hh\lib\copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "D:\Anaconda\envs\test_hh\lib\copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "D:\Anaconda\envs\test_hh\lib\copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "D:\Anaconda\envs\test_hh\lib\copy.py", line 215, in _deepcopy_list
    append(deepcopy(a, memo))
  File "D:\Anaconda\envs\test_hh\lib\copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "D:\Anaconda\envs\test_hh\lib\copy.py", line 280, in _reconstruct
    state = deepcopy(state, memo)
  File "D:\Anaconda\envs\test_hh\lib\copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "D:\Anaconda\envs\test_hh\lib\copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "D:\Anaconda\envs\test_hh\lib\copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "D:\Anaconda\envs\test_hh\lib\copy.py", line 215, in _deepcopy_list
    append(deepcopy(a, memo))
  File "D:\Anaconda\envs\test_hh\lib\copy.py", line 161, in deepcopy
    y = copier(memo)
  File "D:\Anaconda\envs\test_hh\lib\site-packages\tensorflow_core\python\ops\resource_variable_ops.py", line 464, in __deepcopy__
    name=self._shared_name + "_copy",
  File "D:\Anaconda\envs\test_hh\lib\site-packages\tensorflow_core\python\ops\variables.py", line 1155, in _shared_name
    return self.name[:self.name.index(":")]

ValueError: substring not found

HunterMcGushion / hyperparameter_hunter

Metaclass conflict with Keras 2.3.0 #199

Error Traceback

Bug Hunt Notes