Open ksexton opened 7 years ago
@ksexton and I noted this would be useful for testing things such as:
cfg
variables are changed (via environment or otherwise).We noted that do_test()
would allow for the user to customize how they preferred to test their application, such as via test
and stat
or by something more such as http://inspec.io/.
This could additionally introduce something like "Test Dependencies" alongside "Build Dependencies" to facilitate the depending on said packages if necessary. These "Test Dependencies" could would not be required during the build or execution steps.
@smith commented in the Habi-chat:
Inspec has a toml parser in its gemfile but it doesn't ship with it. You could totally have your specs inspect and modify the hab ring and do stuff like (psevdocode):
describe "my-service" do
with_config do |cfg|
cfg.port = 1000
end
it "has the port open" do
expect(cfg.port).to be_listening
end
end
I like the idea of a do_test() implementation
Another thing that would be nice to either test by default or have a built-in helper method available to invoke from tests is running ldd
over all binaries in the build output, checking that no shared library links are broken, and checking that all linked libraries are listed in pkg_deps
This was discussed more heavily in #1470. One comment from that thread was that some packages, like core/jre8
, have some cruft binaries in their output you won't care about leaving broken. I suggest that in that case a plan author should either have their do_install
implementation delete the binaries they don't intend to support before the test runs, or have to override do_links_test() { return 0 }
to skip that test
This would make the habitat build process easier to learn since you could get clear errors about common mistakes rather than "not found" errors on a binary that's clearly there down the line, and it would increase the overall quality and reliability of packages
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. We value your input and contribution. Please leave a comment if this issue still affects you.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. We value your input and contribution. Please leave a comment if this issue still affects you.
CC @eeyun @Hoverbear
Currently Habitat does not support testing that the package created matches what the package author intended. As a package author, I would like to have the ability to ensure that plan updates do not break things that I have specified in my tests.
Some possible ways this could be accomplished:
do_test()
blockhab pkg test
orhab plan test
This could also be useful in a build pipeline. A package could build successfully but not pass the test and be kept from creating the artifact.