[AWS DB] Design architecture for analytics database

[adammy123]

Description With metadata currently being stored in a text file, it requires more effort for analytics. The goal is to have all metadata in a database for easy visualization of data.

Planned Actions Research on the best AWS database solution and design a first draft to store metadata.

[adammy123]

AWS does not support a trigger to add an item to DyanmoDB from an S3 event or an Sns notification. It is only able to trigger a lambda function from a DynamoDB event.

However, there is unity aws sdk that supports adding an item to DynamoDB directly from Unity.

(it is basically free to update the table)

For @chenchen2015 to take further action.

[chenchen2015]

@adammy123 Maybe we can do this instead: S3 -> SNS -> Lambda -> DynamoDB

According to this video, it's possible to append new entries to DynamoDB from a Lambda function

[adammy123]

To-do

• [ ] Make lambda function flexible to new metadata key-value pairs