search

adriangb / scikeras

Scikit-Learn API wrapper for Keras.
https://www.adriangb.com/scikeras/
MIT License
242 stars 50 forks source link

String "loss" being passed to metric_name() instead of loss func name #313

Open Zanchet2020 opened 9 months ago

Zanchet2020 commented 9 months ago

I'm new to ML and I'm just following a tutorial right now, but I'm getting the error:

ValueError: Could not interpret metric identifier: loss

at: \keras\src\metrics\__init__.py:205, in get(identifier)

It seems like in the wrapper.py line 532 metric_name(key) should receive the name of a loss function, but it's actually receiving the string "loss"

If inside keras_metric_get I manually set identifier = "binary_crossentropy" it works fine

adriangb commented 9 months ago

Would you like to make a PR with a test and a fix since it sounds like you've already figured out where the problem is?

Zanchet2020 commented 9 months ago

I can try, though I'm not sure I'm capable of doing it lol

adriangb commented 9 months ago

Give it a shot! Feel free to ask questions.

CasperKjaerhus commented 8 months ago

I am facing this exact issue as well using keras 3.0.5/tensorflow 2.16.1

johloe11 commented 8 months ago

Looks like the issue is with how the _fit_keras_model() method in the wrappers.py file tries to filter out Keras keys. The value error seems to have changed. if you change line 535 from if "Unknown metric function" not in str(e): to if "Unknown metric function" not in str(e) and "Could not interpret metric identifier" not in str(e): it works as expected.

adriangb commented 8 months ago

This is definitely a bug. I’ll try to fix it this week, but PRs are always welcome. I’d suggest a more robust pattern (if key == “loss” or key[:4] == “val_” or similar, suggest looking into keras docs).

lazarust commented 8 months ago

Is someone already working on fixing this? I'm now running into it too

adriangb commented 8 months ago

If you're running into this issue I'd guess you're using an unsupported version of Keras (e.g. Keras 3.0). We don't pin upper versions (which is considered best practice). It's an unfortunate tradeoff you need to make as a Python maintainer.

For now you'll either need use a supported version of Keras / Tensorflow (e.g. Tensorflow 2.12.X without the standalone Keras package) or wait for https://github.com/adriangb/scikeras/pull/317

adriangb commented 7 months ago

Could you please try #317 (pip install -U 'git+https://github.com/adriangb/scikeras.git@keras-3') and see if it fixes your problem? Thanks!