But in reality, setting up platform in the lab typically involves of more activities. Each of them should be verified by such a suite, so it can be used as a way to automatically verify the task of setting up new platform in the lab. This suite can also be started before regression testing to ensure the platform is ready for testing. Nowadays, we often find some setup is missing during regression testing. Detecting problems later in the release process is counterproductive and costs us more in the end. We should focus on detecting problems as early as possible.
What should be verified may be platform dependent, but typically it involves the following:
[ ] Power control - both relay/sonoff and power/reset buttons if avilable
[ ] Serial / PiKVM - depending on DUT_CONNECTION_TYPE
[ ] external flashing
[ ] OSes boot (typically Ubuntu and Windows) - including proper credentials / hostname verification; in some cases for the new platform we do not have Dasharo firmware yet, so this must be done manually still
Further step would be to verify hardware matrix as well, but this is big enough of a task to be tracked separately.
We have something like: https://github.com/Dasharo/open-source-firmware-validation/blob/develop/dasharo-compatibility/basic-interfaces.robot - we could extend it, or add another suite which extends it.
But in reality, setting up platform in the lab typically involves of more activities. Each of them should be verified by such a suite, so it can be used as a way to automatically verify the task of setting up new platform in the lab. This suite can also be started before regression testing to ensure the platform is ready for testing. Nowadays, we often find some setup is missing during regression testing. Detecting problems later in the release process is counterproductive and costs us more in the end. We should focus on detecting problems as early as possible.
What should be verified may be platform dependent, but typically it involves the following:
Further step would be to verify hardware matrix as well, but this is big enough of a task to be tracked separately.