A unit test tests the behaviour of a piece of code. When that code includes a dependancy, it is not the job of the unit test to ensure the implementation of the external lib is correct - only that it's API is behaving correctly within the current program's context.
Does this still hold true if the dependancy is a trained model? What if this model continues to train and evolve in production? Is it enough to know the model still passes the assumptions we've explicitly defined in our unit tests, even if the deeper assumptions the model's behaviour is based on have possibly dramatically shifted?
If we do not know the core assumptions of the model, but only it's behaviour it the explicit tests we've constructed, then is this a potential regulatory risk? Say, for example, you've deployed self-driving car software that has resulted in a fatality because your model hit a particular environmental case that was previously not explicitly tested.
How do we comply to regulations? Is the expectation here that you'd test for every possible circumstance? If so, does that not undermine the flexible problem solving ML promises - creating essentially a procedural program by proxy of the unit tests?
Versioning is yet another issue. Should self learning models be treated as 3rd party APIs rather than imported libs?
In short, there is a tension between leveraging the flexibility of models and embedding their desired behaviour in a program - and being ultimately responsible for the unknown: the inner workings of the model.
A unit test tests the behaviour of a piece of code. When that code includes a dependancy, it is not the job of the unit test to ensure the implementation of the external lib is correct - only that it's API is behaving correctly within the current program's context.
Does this still hold true if the dependancy is a trained model? What if this model continues to train and evolve in production? Is it enough to know the model still passes the assumptions we've explicitly defined in our unit tests, even if the deeper assumptions the model's behaviour is based on have possibly dramatically shifted?
If we do not know the core assumptions of the model, but only it's behaviour it the explicit tests we've constructed, then is this a potential regulatory risk? Say, for example, you've deployed self-driving car software that has resulted in a fatality because your model hit a particular environmental case that was previously not explicitly tested.
How do we comply to regulations? Is the expectation here that you'd test for every possible circumstance? If so, does that not undermine the flexible problem solving ML promises - creating essentially a procedural program by proxy of the unit tests?
Versioning is yet another issue. Should self learning models be treated as 3rd party APIs rather than imported libs?
In short, there is a tension between leveraging the flexibility of models and embedding their desired behaviour in a program - and being ultimately responsible for the unknown: the inner workings of the model.