Open Lunatik00 opened 2 years ago
well, trial and error, it finally worked
model.save(os.path.dirname('model.h5'))
but i get the following warning
2021-11-10 00:53:53.232095: W tensorflow/python/util/util.cc:348] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.
WARNING:absl:Found untraced functions such as dense_layer_call_fn, dense_layer_call_and_return_conditional_losses, dense_layer_call_fn, dense_layer_call_and_return_conditional_losses, dense_layer_call_and_return_conditional_losses while saving (showing 5 of 5). These functions will not be directly callable after loading.
/usr/local/lib/python3.7/dist-packages/keras/utils/generic_utils.py:497: CustomMaskWarning: Custom mask layers require a config and must override get_config. When loading, the custom mask layer must be passed to the custom_objects argument.
category=CustomMaskWarning)
I am not sure if this affect in any way the usability of the network.
I ran into same issue. I tried your workaround os.path.dirname, some h5 seemed to be saved. but i also go this error:
serialize_concrete_function(concrete_function, node_ids, coder) 66 except KeyError: 67 raise KeyError( ---> 68 f"Failed to add concrete function '{concrete_function.name}' to object-" 69 f"based SavedModel as it captures tensor {capture!r} which is unsupported" 70 " or not reachable from root. "
KeyError: "Failed to add concrete function 'b'__inference_similarity_model_layer_call_fn_145923'' to object-based SavedModel as it captures tensor <tf.Tensor: shape=(), dtype=resource, value=
Update: My particular model has some data augmentation layers in them, once i remove them, the model is saved. There may be serialization related issue. One such layer is like:
tensorflow.keras.layers.experimental.preprocessing.RandomFlip
Actually, I discovered this is indeed a bug, that is distinct from inability to save in .h5 format (which may necessitate more code to save the _index?). The error message I got was for saving with the expected 'tf' format.
_Trackable Python objects referring to this tensor (from gc.get_referrers, limited to two hops): <tf.Variable 'random_crop1/cond/Variable:0' shape=(3,) dtype=int64>
This happens for EfficientNetSim (which is used in supervised_visualization.ipynb)
embedding_size = 128 #@param {type:"integer"}
model = EfficientNetSim(train_ds.example_shape, embedding_size)
What i found is a bit surprising. The default for augmentation is "basic" (not nothing), and it involves:
augmentation_layers = tf.keras.Sequential([
layers.experimental.preprocessing.RandomCrop(img_size, img_size),
layers.experimental.preprocessing.RandomFlip("horizontal")
])
So when i used this same model in work, I didn't know it has applied these augmentations. I am biased to think default should be None, since this is highly data-centric and task-centric. Eg. one may argue you shouldn't do this for x-ray if the disease doesn't have left/right symmetry. Model summary sort of gives a high level sequential layer and i miss this as a result.
I suspect if i don't use augmentation, the model will save correctly (avoid using RandomFlip). I will try this and update more if that doesn't resolve.
Finally, for those who really want to save and not redo expensive training, you can try model.save_weights(...) and load_weights(...) as a tmp workaround.
Let me know if this should be a separate issue.
I found it is possible to save if I don't use augmentation in EfficientNetSim.
model = EfficientNetSim(train_ds.example_shape, embedding_size, augmentation=None)
tf.keras.models.save_model(model, "./sim_model.h5")
# OR
tf.keras.models.save_model(model, "./sim_model/0")
We just pushed 0.15 to the main branch and removed the augmentation arg from all the tfsim.architectures. This is a breaking change but should resolve some of these issues.
Hi, i have tried to save the model as shown in the tensorflow page https://www.tensorflow.org/tutorials/keras/save_and_load and i get the following error
tensorflow.python.framework.errors_impl.FailedPreconditionError: model.h5 is not a directory [Op:WriteFile]
I can save it in the other way, is the h5 format that is not working, and it would be better for what i need if i could save it in just one file.
The version are tensorflow==2.6.0 h5py==3.1.0 pyyaml==6.0 tensorflow_similarity==0.14.8
I think it could be a versions issue