Enrich the conversations data with sentiment/question/emotion

[toddysm]

For each message in the conversation, we would like to add additional metadata if the message is:

• Question
• Positive/Negative/Neutral as sentiment
• What is the emotion

Here is an example:

• TeacherFeedback [question:positive:happy]
  • StudentReply01 [statement:neutral:happy]
  • StudentTeacherReply02 [question:negative:sad]

[ZenSorcere]

Initial Estimates

• Planning (issue writeup, initial rough time estimations): 30 min
• Research (S3 access/updating, Python json file updates, Python syntax/methods, inclusion of additional file access and compiling information for json inclusion): 5 hours
• Implementation and Testing (code writing/optimizing, pulling information from multiple sources, testing adding of data to existing s3 object, connection/uploading to s3, error handling): 5 hours
• Documentation (writeup code explanation and how-to for unfamiliar user): 1 hour
• Deployment (PR Write-up): 1 hour

[ZenSorcere]

Enrich Conversation data with sentiment/question/emotion metadata

Pre-Requisites:

• Imports for boto3 and nltk
• AWS credentials saved in ~/.aws/credentials file

Implementation:

• Script accesses the S3 bucket and the auto-floop-s3-export3.json file. Data for all json entries are pulled. To speed up verification, a list of 50 conversations are taken for analysis and put into a List.
• For each conversation in the list, for each message in the conversation, the Text value is retrieved and run through three functions:
  • sentAnalyze (determines "question/sentence" state)
  • sentence_classification (determines "positive/negative/neutral" state)
  • emoAnalyze (determines "emotion" state)
• Each message dictionary is appended with a new K:V pair: "metadata": ["question", "posNeg", "emotion"]
• The updated conversations are then put into a new json file (enriched-conv-sample.json), and the json file object is added to the designated s3 bucket .

Input:

• script execution, which pulls in json file from s3.

Output:

• new json object, enriched-conv-sample.json, in the designated s3 bucket.

[ZenSorcere]

Activities Documentation

Date	Activity	Time Spent
2/23/22	Planning	15 minutes
	Research	1 hour
	Implementation/ Testing	1.25 hours
	Task Management	15 minutes
2/24/22	Research	1.75 hours
	Implementation/ Testing	30 minutes
2/25/22	Implementation/ Testing	2.75 hours
	Documentation	15 minutes
	Deployment	1 hour
	Task Management	30 minutes
3/01/22	Deployment Updates	30 minutes

Time Totals

Activity	Time Spent
Planning	.25 hour
Research	2.75 hours
Implementation/Testing	4.25 hours
Task Management	.75 hour
Deployment	1.5 hours
Documentation	.25 hours
TOTAL TIME:	9.75 HOURS