search

overthesun / simoc-sam

Live backend for SAM at Biosphere 2
2 stars 1 forks source link

Integrate Cassandra workflow into sensor classes #28

Open IMBCIT opened 2 years ago

IMBCIT commented 2 years ago

Cassandra integration

Current working branch is cassandra_sam.

Removing all CQL query functions in favor of using Models exclusive is probably the best path forward.

A .create() function will need to be called on the model in question to create the database entries. There is also the ability to batch write them after intervals to cut down the need to collect data as often.

Examples can be seen here

Database Design Options

In order to ensure an unique UUID is assigned to each sensor we can either:

  1. Generate a UUID4 when the sensor is given a unique name in python code and use this as the UUID.
    • Potential issue if sensor drops off and does not retain a unique name another UUID will be generated, thus providing two instances of a sensor there there should be one
    • This requires us to have a solid way of ensuring there is a set unique name/id already on the sensor which is linked to issue #27
  2. Generate a UUID4 for the sensor based on the .cfg or .yaml file modified by the team and store it in the config as a reference for data integrity
    • Having multiple of the same sensor on a device/raspberrypi can become possibly confusing to differentiate which one is which one in case they go down.
    • Relies on the .cfg files to be properly maintained which may put undue pressure on the team to ensure that the data is accurate
  3. Generate a UUID4 for sensor in the read_sensor_data function
    • This is the easiest solution but if this function fails or needs to be restarted a new UUID will be created each time. Might be best to test with this before a better solution is found for the unique naming and retention on the sensors. This may or may not be an issue with the Vernier sensors as the API might have native support for a unique name/ID that is immutable.