Implementing DynamoDB and Boto3

[AyeZian]

Plan out how we will be using DynamoDB and Boto3, specifically how we will be storing data.

[MarkKoz]

Some things to look at:

• Set up a mock Db with DynamoDB Local.
• Sensor data is time series data since we want to keep historical data around too. AWS has an article on this.
• The backend may need to know when a new record is inserted into DynamoDB. Streams can help with this. My idea was to use this for implementing the notification system. The backend would listen for database events and check it against all known notification conditions. If an event satisfies a condition, then the backend will send out a notification to the user (or ask the frontend to show a notification).
• Sensor data needs to be associated with the test history table in the backend. Each row has a unique ID, so DynamoDB can just associate its records to that ID. The hope is that the hardware will be able to receive this ID when it's requested to run a scheduled test. The problem is that DynamoDB isn't a relational database. However, we did find an article discussing ways around this. Denormalization by duplicating data may particularly be of help (though superficially, duplication sounds like a bad thing).

I'll add another comment with more things if I think of anything else later.

[MarkKoz]

We decided to only support the same 4 sensors for all devices. Furthermore, we decided for now to store the data for each sensor as a separate table. Thus, there will need to be 4 tables. Each table needs to have a test ID and the sensor data resulting from that test.

We discussed adding the insertion time as a column to each table. It's not completely clear if this will be necessary, as each test ID already has a specific time it's associated with in the backend. Having the device store the insertion time may be useful for determining when the test finished. However, if the backend uses a stream to get notified of insertions, it could also just take the current time at the point of receiving a notification from the stream

In any case, it probably doesn't hurt too much, if at all, even from a good-practice design perspective, to include the time as a column. One possible argument against it could be that both DynamoDB and the PostgresSQL db for the backend would be storing the same data, since the latter's model has a field for the test's end time.