Open ieugen opened 4 years ago
rouazana
commented
4 years ago Did you have a look at https://github.com/apache/james-project/blob/master/dockerfiles/compilation/java-11/integration_tests.sh which provides some basic smoke tests for a built server?
chibenwa
commented
4 years ago Did you have a look at https://github.com/apache/james-project/blob/master/dockerfiles/compilation/java-11/integration_tests.sh which provides some basic smoke tests for a built server?
It misses some of the point mentioned above like JMAP, SMTP.
https://github.com/chibenwa/james-blacklist/tree/master/blacklist-integration could provide a way to write Java integration tests (and covers extension testing). However James would be started in the testing JVM, while the link pointed by @rouazana relies on a dockerized James server.
ieugen
commented
4 years ago Thanks for the links. I will check them out.
I would like to test the released artifact. If possible I would like to interact only with the exposed API and not use any back-channels.
In this context the docker image or the files built with tar.gz is the relased artifact.
I will make configuration changes and code changes to add extension points. I would like to be able to check that system produced by the build is working properly.
For that I believe we need to have some form of testing. We should test for stuff like:
At this point, I expect to automate just the running part. Checking for fail/pass can be done by going through the logs with a written checklist like the one above. The logs should contain clues that each sub-system works as expected. It would be great if some checks can be automated, but I believe some of them will require a lot of work to do and it will not be worth it.
In time we might include a smoke test API for these ?!
Looking for docs on how to work on each one.