This PR adds helper classes that make it easier two write upgrade tests. Instead of writing the same test twice, once for a single device with the current firmware and once with a preparation step with the old firmware and a verification step with the current firmware:
This will automatically generate two test functions like these:
def test_exec(self, device: Device) -> None:
with self.context(device) as context:
state = self.prepare(context)
self.run(context, state)
@upgrade
@pytest.mark.virtual
def test_exec_upgrade(self, serial: str, ifs: str) -> None:
with spawn_device(serial=serial, ifs=ifs, suffix="old") as device:
with self.context(device) as context:
state = self.prepare(context)
with spawn_device(serial=serial, ifs=ifs, provision=False) as device:
with self.context(device) as context:
self.run(context, state)
The big advantage of this approach is that we can easily test upgrade stability and data retention for all features. The downside is that we have to change the scope of the device fixture from module to function because the upgrade tests are no longer in a separate module. That means we have to spawn a new virtual device for every test function, adding an overhead of up to five seconds per test.
Which way do you prefer? Or do you have any ideas how to avoid the overhead?
This PR adds helper classes that make it easier two write upgrade tests. Instead of writing the same test twice, once for a single device with the current firmware and once with a preparation step with the old firmware and a verification step with the current firmware:
We can now write it only once:
This will automatically generate two test functions like these:
The big advantage of this approach is that we can easily test upgrade stability and data retention for all features. The downside is that we have to change the scope of the
device
fixture from module to function because the upgrade tests are no longer in a separate module. That means we have to spawn a new virtual device for every test function, adding an overhead of up to five seconds per test.Which way do you prefer? Or do you have any ideas how to avoid the overhead?