rasbt / python-machine-learning-book-3rd-edition

The "Python Machine Learning (3rd edition)" book code repository
https://www.amazon.com/Python-Machine-Learning-scikit-learn-TensorFlow/dp/1789955750/
MIT License
4.6k stars 1.99k forks source link

Chapter 16 - tfds.feature.text module is deprecated. #155

Closed Jeffresh closed 3 years ago

Jeffresh commented 3 years ago

In page 587: you use :

Dont know which is the new implementation.

Jeffresh commented 3 years ago

This happens in all the following code that use feature.text module and is solved using the deprecated statement. I try to find where this modules are moved, but I can't find it. I find that now tensorflow has tensorflow_text module, a dedicated module for preprocessing text, that you have to install separately, like dataset module, but the logic , method's names and use of them are changed a lil bit, so don't now the cost to migrate the code to use that.

tensorflow_text official doc

rasbt commented 3 years ago

Thanks for the note! Just updated it so that it should work with newer tensorflow_dataset versions via

try:
    tokenizer = tfds.features.text.Tokenizer()
except AttributeError:
    tokenizer = tfds.deprecated.text.Tokenizer()