Open DominicOram opened 3 weeks ago
My two cents are:
dev
instance that is well maintained by the ispyb team and so require little overhead by us to maintainMy main concern here is that the continuous failure of s03 tests (and our collective reluctance to maintain them) is causing us to miss potentially useful errors in the tests that we can more easily maintain or causing us to not bother maintaining any of them. I propose we remove the tests against s03 and depreciate s03 as a concept in it's current form. However, I could be open to at least splitting the s03 tests off into a separate set of system tests.
Things that would change my mind:
eiger
(thanks to @GDYendell) which I think would be good to try and run some system testing against. However, it's not clear that we have that for other devices or if there's a path to having itA similar discussion should be had with dodal
I agree with your comments about s03. I also think we should make clear which systems tests should 'just work' on a diamond workstation with no extra steps, as those are the most useful ones.
I've been ignoring most of the system tests because I keep expecting none of them to ever work, but like you say, some are actually very useful!
See https://github.com/DiamondLightSource/dodal/pull/777 for an example of a useful test that we're currently missing out on
With the move done in https://github.com/DiamondLightSource/mx-bluesky/pull/172 many of the system tests are now not working. In reality many of the system tests have been not working for some time but have not been maintained. We should have a discussion about why this is. Is the maintenance overhead of the system tests worth the benefit they give us?
Acceptance Criteria