Step One

[derekstephen]

======

Calculates TF-IDF of mission statement.

Code

• [x] Format Mission Statements (https://github.com/derekstephen/AidlyAI/pull/2)
• [x] Snowball Stemmer (https://github.com/derekstephen/AidlyAI/issues/3#issuecomment-599400242)
• [x] POS (Requirement for Lemmatization) (https://github.com/derekstephen/AidlyAI/issues/3#issuecomment-599426648)
• [x] Lemmatization (https://github.com/derekstephen/AidlyAI/issues/3#issuecomment-599426648)
• [x] Calculate TF-IDF

After

• [x] Table with: Term, Frequency, Document Frequency, Overall Frequency
• [ ] Word Cloud

[derekstephen]

Used Snowball Stemmer instead of Porter Stemmer due to nltk website stating english Snowball Stemmer better than porter Stemmer.

• Link
• Push https://github.com/derekstephen/AidlyAI/commit/891421d72cced307b183abae788b9d6b2c8f814a
• Code Reference https://github.com/derekstephen/AidlyAI/blob/891421d72cced307b183abae788b9d6b2c8f814a/step_one.py#L48-L56

[derekstephen]

Inquiry: Lemmatisation in code. Would it require POS or setting as Verb enough?

• Link 1
• Push https://github.com/derekstephen/AidlyAI/commit/741ff22d80cfc0dfa9d2b1be11c4e1a2bdaab2df
• Code https://github.com/derekstephen/AidlyAI/blob/741ff22d80cfc0dfa9d2b1be11c4e1a2bdaab2df/step_one.py#L62-L66

[derekstephen]

Set up Lemmatizer

• Push https://github.com/derekstephen/AidlyAI/commit/a24189a1f6729a6026deecf9284270d9ef36952e
• Code https://github.com/derekstephen/AidlyAI/blob/a24189a1f6729a6026deecf9284270d9ef36952e/step_one.py#L51-L91