Do you prefer to filter tests by tags or file names?
Looking at check_build_push_template.yml I see you have the workflow variable ${{ matrix.image }}, so you can either tag your tests with the image name and then let robot find the tests with that tag, or you could have a suite file specific to the image and just call the robot file.
BTW what @KseniaYa did here to create the ${{ matrix.image }} is quite cleaver :+1:
Advantage of the tag method is you can potentially re-use some tests for multiple images (just put all relevant tags on that test case) but there might not be a need for this either, e.g. if an image is based on another image, e.g. sshlibrary is based on base, there's no need to re-run the tests that apply to base on sshlibrary, as they were already run on base.
The other thing to think about is do you run robot inside the build job (check_docker_build) or do you create a new job in between check_docker_build and push_to_registry?
Anyway that's my initial thoughts on this, hopefully gives you some food for thought,
Do you prefer to filter tests by tags or file names?
Looking at check_build_push_template.yml I see you have the workflow variable
${{ matrix.image }}
, so you can either tag your tests with the image name and then let robot find the tests with that tag, or you could have a suite file specific to the image and just call the robot file.BTW what @KseniaYa did here to create the
${{ matrix.image }}
is quite cleaver :+1:FYI tag method would look something like:
FYI file method would look something like:
Advantage of the tag method is you can potentially re-use some tests for multiple images (just put all relevant tags on that test case) but there might not be a need for this either, e.g. if an image is based on another image, e.g.
sshlibrary
is based onbase
, there's no need to re-run the tests that apply to base on sshlibrary, as they were already run on base.The other thing to think about is do you run robot inside the build job (
check_docker_build
) or do you create a new job in betweencheck_docker_build
andpush_to_registry
?Anyway that's my initial thoughts on this, hopefully gives you some food for thought,
Dave.