danielwatson6 / skip-thoughts

Simple TensorFlow implementation of skip-thought vectors
Do What The F*ck You Want To Public License
11 stars 4 forks source link

Add script to clean training data #1

Closed danielwatson6 closed 6 years ago

danielwatson6 commented 6 years ago

Tokenization is necessary, but preprocessing should be kept minimal for simplicity and model robustness.

Documentation on script should be added to the README.