search

rasbt / mlxtend

A library of extension and helper modules for Python's data analysis and machine learning libraries.
https://rasbt.github.io/mlxtend/
Other
4.86k stars 857 forks source link

Error while running ```bias_variance_decomp``` for Tensorflow-Keras Functional API #759

Open jackaraz opened 3 years ago

jackaraz commented 3 years ago

Hi All

Hope you are doing fine in these strange times. I got a problem with bias_variance_decomp while using it for tf.keras.models.Model. The problem was coming from /mlxtend/evaluate/bias_variance_decomp.py L110. First of all, for some reason my model can't pass if estimator.__class__.__name__ in ['Sequential', 'Functional'] so it goes through else at L108 where code takes pred = estimator.fit(...).predict(...) but estimator.fit is just history object apparently and it doesn't have predict functionality, so I patched it as follows

        else:
            estimator.fit(X_boot, y_boot, **fit_params)#.predict(X_test)
            pred = estimator.predict(X_test).reshape(1, -1)

but wanted to run by you to see if it might create any problem in the future or if I'm missing anything.

Python version 3.6 Tensorflow v2.1 (Keras within)

Thanks Cheers Jack

rasbt commented 3 years ago

Hm, good question. The example in the documentation (http://rasbt.github.io/mlxtend/user_guide/evaluate/bias_variance_decomp/#example-3-tensorflowkeras-support) was ran with having TensorFlow v2.3.1 installed on my machine and only tried with Sequential. Maybe 

if estimator.__class__.__name__ in ['Sequential', 'Functional']

is not backwards compatible with earlier versions of Tf or it doesn't work for tf.keras.models.Model. Maybe @hanzigs knows a little bit more and could help

hanzigs commented 3 years ago

Yes, else part is not for tf keras model, and I worked that with Tensorflow 2.3.0. Have to check this estimator.__class__.__name__ for earlier versions.