Investigate Ways to Communicate with Hardware

[MarkKoz]

The current vision is as follows:

• When a device is registered, it will send the backend information about what kinds of sensors it has. The backend doesn't necessarily need to understand the semantics of these sensors; it could just be any arbitrary identifier, as long as that device will understand what to do if it was told to perform a test using that sensor.
  • Will need to facilitate the storage of the supported sensors in the backend.
    • If it's a one-to-one relationship with parameters, it could just be stored in a parameter model, and the user would be required to fill in all parameters as the device is registered.
  • May need a way to support re-reporting supported sensors in case the sensors are swapped out after initial registration.
• The user will be able to create tests per-device using the pool of reported test types for that device.
• When it's time for a test to run, the backend will send a message to the hardware that contains the sensors to use and a unique ID that will should be associated with the gathered sensor data.
• The device needs some way to report test status (e.g. success or failure) to the backend. Not sure if this should be a separate message sent to the backend, or if it would instead work out nicely to just include the status in dynamodb (but then we're storing the status both in dynamodb and in the backend's DB, which is not ideal).

The question is: how can the backend communicate with the device? Can we leverage AWS for this? This is a quite important aspect of the project that needs to be investigated.

[MarkKoz]

In yesterday's meeting, we decided to simply the plans. There will only be 4 sensors supported, and these are predetermined. Thus, for all devices that get registered, the backend will just assume each device has those 4 specific sensors. This removes the need for a device to report which sensors are available.

We're still thinking of a way to have the device report the test status to the backend. One way could be to add an extra column to the table that indicates the status. Then, the backend could rely on streams to be notified of when the row is inserted, and update its own model with the status. However, this isn't ideal for two reasons:

1. Data (the status) is being duplicated across two different databased (DynamoDB and PostgresSQL). There is nothing enforcing their consistency, though in practice, I suppose once the status is stored in PostgresSQL, the value of DynamoDB no longer matters. However, from a design perspective, this is poor practice. There's nothing restricting someone from later reading from DynamoDB, even if that's not what they should be doing.
2. If no data is produced, then the sensor data column would have to be null, since the sensor data and status would be in the same row along with the test ID.

We can look into more direct ways of communication. One thing to keep in mind is robustness i.e. ensuring that the device can report the status of the test even if something crashes or goes really wrong. The backend totally relies on the device for this information - it has no way to determine on its own if there's a failure, unless it uses some time-based heuristic.

[MarkKoz]

Update on the state of things:

The Django backend will communicate with devices using MQTT. Each device will be subscribed to a topic, and the backend will use the Python AWS SDK to publish messages on that topic. The messages are simply in JSON format. The backend will send a message consisting of the test history ID and the sensor ID. The sensor ID is an ID which all devices consistently recognise as mapping to the same sensor (e.g. no matter what device receives the message, an ID of "temp" will always be understood as the temperature sensor).

Current plan for sending these messages from the backend is to use Django Q, which is a task queue library for Django. It will be used to queue tasks to run "in the background" at some specified point. Specifically, it will be used to schedule the sending of the message to the device, so the device is notified that it should start some particular test.

Something that still needs to be figured out is whether each device will need to be subscribed to a unique topic. Furthermore, will it be possible to prevent a device from receiving messages meant for a different device, or at least for devices owned by a different user?

Regarding more direct communication, the backend could also subscribe to an MQTT topic to which a device can publish to. This would require an MQTT client like Paho or the one included in the AWS IoT Device SDK v2 for Python (boto3 does not have an MQTT client). In fact, we probably could have used this instead of DynamoDB streams or DynamoDB overall. In either case, the question stands of how to trust a device is the one it says it is. Might need to look into AWS IoT certs in more detail. This is not high priority though for an MVP.