Is your feature request related to a problem? Please describe.
When using Loom in a research context, it is assumed that if a feature is integrated into Loom, that same feature can be deployed and will continue to function for an indefinite period of time. Currently Loom does not perform any long-term testing of integrated features, and as a result cannot provide any such guarantee. While this lack of verification may have aligned with the original goals of Loom (education about sensors), it has become apparent that a long-term guarantee will be necessary if Loom continues to move towards being a pure-research platform.
When speaking with researchers, we found a reasonable time-frame for a remote deployment to be ~6 months. Not all functionality in Loom is worth verifying, but at least a minimum viable products worth should be verified to function without human interaction for 6 months.
Describe the solution you'd like
I propose that every month or major feature change, a known subset of Loom functionality (such as the SmartRock project with a battery) be built and flashed (and labeled) with the current stable Loom, and placed into a cabinet. Each week (or automatically) this build of Loom should be checked to verify it is behaving correctly. The next month/major feature, this process should be repeated, placing the new build next to the one from the previous month, and so on. Once a build of Loom passes the six month mark, that build of Loom is considered "proven" and can be removed from the cabinet. Once a build of Loom is placed into the cabinet, it's code in source control and it's attached peripherals must not be modified, otherwise jeopardizing the test results.
With this system, after the initial six months a build of Loom will be considered "proven" once every month, allowing projects in active deployments to still receive features on a longer timescale. If a project uses a feature from Loom that has not been "proven", we will make no guarantee of the feature working beyond a few days.
Is your feature request related to a problem? Please describe. When using Loom in a research context, it is assumed that if a feature is integrated into Loom, that same feature can be deployed and will continue to function for an indefinite period of time. Currently Loom does not perform any long-term testing of integrated features, and as a result cannot provide any such guarantee. While this lack of verification may have aligned with the original goals of Loom (education about sensors), it has become apparent that a long-term guarantee will be necessary if Loom continues to move towards being a pure-research platform.
When speaking with researchers, we found a reasonable time-frame for a remote deployment to be ~6 months. Not all functionality in Loom is worth verifying, but at least a minimum viable products worth should be verified to function without human interaction for 6 months.
Describe the solution you'd like I propose that every month or major feature change, a known subset of Loom functionality (such as the SmartRock project with a battery) be built and flashed (and labeled) with the current stable Loom, and placed into a cabinet. Each week (or automatically) this build of Loom should be checked to verify it is behaving correctly. The next month/major feature, this process should be repeated, placing the new build next to the one from the previous month, and so on. Once a build of Loom passes the six month mark, that build of Loom is considered "proven" and can be removed from the cabinet. Once a build of Loom is placed into the cabinet, it's code in source control and it's attached peripherals must not be modified, otherwise jeopardizing the test results.
With this system, after the initial six months a build of Loom will be considered "proven" once every month, allowing projects in active deployments to still receive features on a longer timescale. If a project uses a feature from Loom that has not been "proven", we will make no guarantee of the feature working beyond a few days.