Open chouseknecht opened 6 years ago
Molecule also uses yamllint internally (as well as ansible-lint), so this isn't exactly a replacement.
What do you want to use molecule for? Full role testing or just some linting?
@paulfantom see revised initial comment. Hadn't really thought through the whole thing until now.
I am doing exactly that in @cloudalchemy roles however not in VM but in docker containers! I think I can share my knowledge if you want.
@paulfantom please do
What form do you want? A video conference call?
Hey, I'm use molecule for all my roles too. I'm ready to share my knowledge with anyone. Also, I use docker (for public) and another drivers (for private) in molecule testing roles
With help from @rcarrillocruz we've learned that Molecule tests, assuming the Docker provisioner, can execute on Zuul with no changes. (Keep in mind, the use fo Zuul as a CI platform is only for partner content.) We also know that Travis can execute Molecule tests as well, assuming the Docker provisioner, with no changes.
Based on these preliminary findings, it seems safe to say that we can move forward with Molecule as our testing framework, and craft some working examples of how both Community and Partner contributors can implement Molecule.
For partners, we may have to be prescriptive about how the tox.ini
file is configured. For example, we may only support testing of recent versions of Ansible, and not go all the way back to 2.0.
On the Galaxy import side we will want to do the following (I think):
Still to be determined... How to use molecule to test a multi-content repository.
I have couple of thoughts about this topic, so I'll just dump them here. Maybe it will be useful for someone.
1) When using docker provisioner you still need to get images from somewhere and usually official docker images of linux distros don't come with init system. This means that using any service management module (ex. systemd
) won't work in such containers. I solved it by delivering my custom images, but I wonder how this will be solved here?
2) What would be run in default
molecule scenario? Role with default variables?
3) "devise a way to inspect the level of testing that's being done - sort of like code coverage" I would love that! However from my experience there is no good automated way of doing that. Molecule supports concept of scenarios and I ended up using two of them: one for role with only default values and second one where everything is changed. But there might be a possibility when there is a need for more scenarios for full code coverage and that usually breaks automated tooling.
4) "inspect the existence of a Molecule directory in the repo" That could be a good start for checking role quality and later allow sorting roles by such metric or just have a filter. It should be fairly simple to implement and have quite large impact on UX.
It'll be up to the content author to provide the tests and test scenarios. If you want to use custom images, that'll work, as long as they're publicly accessible.
FWIW, we're using a copy of your grafana role for our POC.
To simplify the lookup of (molecule) tests presence, it might be a good idea to provide a way to specify a path to them and maybe a command for running those in the metadata.
Molecule tests are quite easy to localize as molecule has quite static directory layout:
molecule/
<scenario_directory>/ <-- usually named "default"
molecule.yml <-- scenario config
{create,destroy,prepare}.yml <-- playbooks to manage test env
playbook.yml <-- actual role execution playbook
tests/ <-- directory with verifier (ex. testinfra) tests
In simplest and most naive way you just need to check if there is a molecule
directory.
Fair enough. If we can force everyone to follow this :)
Add to molecule
directory to ansible-galaxy init
and hope for the best? :smile:
Create a POC that includes the following:
molecule
directory structuremolecule
secnariosFrom the POC we want to learn the following: