Find and implement a testing framework

[TheHackmeister]

Start by looking for a framework for testing MQTT in some way. If that doesn't exist, look for something more generic; a framework for API testing might be usable. Alternatively, there might be value in a framework for testing microservices or container networks.

One way to test TechnoCore is to create a test swarm with fake devices outputting and receiving MQTT messages.

[TheHackmeister]

It makes sense to try and use the same code for testing as for health checks: #7

[TheHackmeister]

Thing to test should include:

• [ ] All of the services start and are initialized correctly: It may be possible to have each service send out a MQTT message once it's initialized and running. Then testing for this is simply a matter of listening for all the services to come up. Perhaps this should be in the installer? I'd like the installer to be able to detect that and report when the application is usable.
  • Docs
  • Vault
  • MQTT (VerneMQ)
  • PlatformIO
  • Portainer
  • Node-RED
  • Home Assistant
  • Home Assistant DB
• [ ] HAL firmware: I can absolutely check that it builds, but more than that I'm not sure how to do. Are the ESP8266 and ESP32 simulators out there? Failing that, I'd consider building a testing rig that would allow me to flash ESPs that have sensors hooked up, and then run those through a series of tests.
• [ ] Home Assistant components: I'd like to provide some kind of test for the following bits. I suspect there is some tests being ran by Home Assistant, and I should figure out how to test these via whatever method is already being used.
  • HAL discovery and entities
  • HAL Lovelace card
• [ ] PlatformIO
  • Firmware builds
  • Tries to flash when MQTT message is received.
• [ ] Node-RED flows: There aren't any yet, so I'm not sure how to test them. I imagine a service that sends out fake HAL data could set up specific scenarios.
• [ ] Documentation:
  • Each link goes to a real page
  • Every image exists
• [ ] MQTT users get created
• [ ] Email is still not spammy

[TheHackmeister]

Some thoughts on behavior driven dev tools: https://opensource.com/article/19/2/behavior-driven-development-tools

[TheHackmeister]

When these are implemented, we should also add badges to the README.md with build status, code coverage... Be thinking about what other badges to include.

[TheHackmeister]

I'd consider using GoCD for the CI/CD runner.