Training Data

[duggurd]

Inspect proposed datasets, download and prep for training. Training data sources.

1. Extract data
2. Transform data
   • Feature engineering
   • Clean data

   • 11 - TF-IDF as a feature?

3. Ingest into database?
4. Store locally?

https://github.com/duggurd/graduation_project/blob/main/src/etl/amazon_reviews_transform.ipynb

[duggurd]

TODO:

• Set max length of text? For less complex models?
• Write into .py file and clean up code
• Better documentation
• More transformations are needed ie. remove punctuation
• Add TF-IDF to data, then training data is finished
• Store trained TF-iDF and word vectorizer in google drive