To help improve the test coverage, I've been directly calling the CRUD operations on resources using resource.TestResourceData().
The issue with using this is that when testing error paths within unit tests and validating the error response. When the acceptance tests are run, it changes the behaviour in these unit tests to panic which then causes those unit tests to fail.
Attempted Solutions
There is a few potential work arounds that could work but aren't really ideal:
Using build tags as part of the acceptance tests
Skip unit tests when acceptance test are running
Seperate the unit tests from acceptance tests
Solution 1 means that when performing test coverage, or trying include unit tests is done within acceptance test coverage that tests required to run twice. This is solution is potentially error prone since it has some knowledge on how the helper libraries works in the two seperate test conditions.
Solution 2 means that there is a manual check for validating if the tests are going to be run in acceptance mode or not. This too is error prone in its behaviour since it also expects the user to be aware of the libraries implementation during acceptance test mode.
Solution 3 from the offset seems to be the most reasonable as a work around but there is currently no guidance to set up the project in this way. There is also the fact that it might not allow some functionality to be exposed for testing when you may want it to be (or require some additional state being passed which could be avoided).
Proposal
The easiest way forward to my understanding would be to remove the panic on error behaviour within the helper libraries to avoid this scenario to begin with. The alternative approach would to provide better guidance on how best to set up a project to allow for unit and acceptance test within the same test invocation.
SDK version
Use-cases
To help improve the test coverage, I've been directly calling the CRUD operations on resources using
resource.TestResourceData().The issue with using this is that when testing error paths within unit tests and validating the error response. When the acceptance tests are run, it changes the behaviour in these unit tests to panic which then causes those unit tests to fail.
Attempted Solutions
There is a few potential work arounds that could work but aren't really ideal:
Solution 1 means that when performing test coverage, or trying include unit tests is done within acceptance test coverage that tests required to run twice. This is solution is potentially error prone since it has some knowledge on how the helper libraries works in the two seperate test conditions.
Solution 2 means that there is a manual check for validating if the tests are going to be run in acceptance mode or not. This too is error prone in its behaviour since it also expects the user to be aware of the libraries implementation during acceptance test mode.
Solution 3 from the offset seems to be the most reasonable as a work around but there is currently no guidance to set up the project in this way. There is also the fact that it might not allow some functionality to be exposed for testing when you may want it to be (or require some additional state being passed which could be avoided).
Proposal
The easiest way forward to my understanding would be to remove the panic on error behaviour within the helper libraries to avoid this scenario to begin with. The alternative approach would to provide better guidance on how best to set up a project to allow for unit and acceptance test within the same test invocation.
References
N/A