search

mrdbourke / tensorflow-deep-learning

All course materials for the Zero to Mastery Deep Learning with TensorFlow course.
https://dbourke.link/ZTMTFcourse
MIT License
5.05k stars 2.5k forks source link

TextVectorizer and .h5 format #632

Open sodeso-nl opened 4 months ago

sodeso-nl commented 4 months ago

In the Tensorflow Developer Exam we will need to save the models in .h5 format, how do we do that for NLP models were we are using the TextVectorizer? Since this layer does not support saving? Will this become clear when doing the exam or is there some trick that is not mentioned in the course?

Or do we need to use a different construction? I mean, we can use the TextVectorizer ouside the model to pre-process all the training data and validation data. But how would that work when submitting the model? Google is probably testing the model with their own sentences, how do they know how to vectorize them to the same numbers.

LubuntuSharp commented 1 month ago

facing the same issue, where you able to find a solution ? thanks,

sodeso-nl commented 1 month ago

So i did the Exam and altough the TensorFlow xam is not available anymore i cannot go into details due to the NDA, however i can point out that in normal circumstances you can for example have the TextVectorizer outside of the model and vectorize the text for example in your data pipeline in a .map function. Then it is not part of the model and it will not cause any issues.

LubuntuSharp commented 1 month ago

Thanks a lot, you help is greatly appreciated! Regards,