External Dependency or Mystery Guest in Test Code

[Talismanic]

Greetings, I am an engineer who is interested in infrastructure as code testing. Currently, I am looking for testing anti-patterns in iac test scripts. I noticed external dependency or mystery guests are occurring in test instances by using an external URL to check the service. As a consequence of this, the test script violates the principle of self-containment. If the domain or uri of the health check changes or /etc/hosts update fails, it will impact the overall test. I think this is an anti-pattern of testing practices.

So I have the following query:

Do you agree that this is an IaC testing anti-pattern? Do you want to fix this?

Any feedback is appreciated.

Source File: https://github.com/OpenConext/OpenConext-deploy/blob/master/tests/all_services_are_up.yml

[thijskh]

Thanks for the effort you’re undertaking. I think it’s a false positive: the *.vm.openconext.org domain is used in the /etc/hosts file of the test machine to point to the machine itself so the urls are actually local to the machine.

[Talismanic]

@thijskh , in that case, in principle do you agree that test scripts specially should not use any external urls which might create dependency on network , existence of the resource on specific urls?

[thijskh]

I would say "it depends". It's definitely something you should make a concious decision to depend on. But e.g. in this repo, we also test a full installation. It means we run some "yum install" commands since this is what the Ansible scripts are made to do. This makes us depend on the availability of the centos repo. I think this is OK since this is what the code is meant to do.

[Talismanic]

Thanks a lot for taking the time. I am closing the issue as I got the insight which I needed.