search

SciSharp / TensorFlow.NET

.NET Standard bindings for Google's TensorFlow for developing, training and deploying Machine Learning models in C# and F#.
https://scisharp.github.io/tensorflow-net-docs
Apache License 2.0
3.17k stars 507 forks source link

[BUG Report]: Loading weights exported from Python doesn't work #1150

Closed OliBomby closed 12 months ago

OliBomby commented 1 year ago

Description

I'm trying to run a model I trained in Python on TensorFlow.NET. I have the weights in HDF5 format and the model replicated in TensorFlow.NET, but when I use model.load_weights(path.h5) it fails to load the weights because it can't find the weights by name.

I did some debugging and found out that it expects dollar signs in the path for some reason, which are normally not there in the weight names.

https://github.com/SciSharp/TensorFlow.NET/blob/fa2d2dcfc894686846f475a98300b6f8af0bfa94/src/TensorFlowNET.Keras/Saving/hdf5_format.cs#L137C21-L141C61

This code seems to be recently added by https://github.com/SciSharp/TensorFlow.NET/pull/1137. The code looks awful, so please revert that change.

Reproduction Steps

In Python with TensorFlow, export some model weights using model.save_weights('my_checkpoint.h5')

Then in C# with TensorFlow.NET, load the weights using model.load_weights("my_checkpoint.h5");

Known Workarounds

Exporting weights using TensorFlow.NET inserts the dollar signs so loading works in that case.

It might be possible to workaround if you manually replace all weight names in the saved weights file.

Configuration and Other Information

Wanglongzhi2001 commented 12 months ago

Hello, thank you for your attention to TensorFlow.NET, this PR does looks have some problems, we will fix it. BTW, model.load_weights does not implemented prefect in TensorFlow.NET, you can try use model.save and keras.models.load_model to save and load the whole model, we recommend you to use this api, because it implemented better in TensorFlow.NET

Beacontownfc commented 12 months ago

Thank you for your attention to TensorFlow.NET, I have restored model.load_weights and improve #1137 .

OliBomby commented 12 months ago

Hello, thank you for your attention to TensorFlow.NET, this PR does looks have some problems, we will fix it. BTW, model.load_weights does not implemented prefect in TensorFlow.NET, you can try use model.save and keras.models.load_model to save and load the whole model, we recommend you to use this api, because it implemented better in TensorFlow.NET

Using keras.models.load_model is what I tried initially but it didnt work because I got a NotImplementedException at https://github.com/SciSharp/TensorFlow.NET/blob/fa2d2dcfc894686846f475a98300b6f8af0bfa94/src/TensorFlowNET.Keras/Saving/SavedModel/RevivedLayer.cs#L41 Turns out my model exported from Python has input_spec specified for all layers.