This PR is auto-generated from #12983 to be assessed for backporting due to the inclusion of the label backport/1.11.x.
The below text is copied from the body of the original PR.
This PR introduces a new model for running blackbox tests with Packer.
By blackbox testing we mean invoking the executable in a realistic but controlled environment, so we can run tests based on a user's perspective, and run shell-like pipelines on the command's outputs.
This allows us to ensure Packer behaves consistently between versions, and makes sure if there's a change in a test, that it is covered and conscious.
Since we're changing how plugins are loaded with Packer 1.11, the tests included in this branch are essentially covering those changes, with eventually more and more tests that can/will be migrated to this architecture.
Backport
This PR is auto-generated from #12983 to be assessed for backporting due to the inclusion of the label backport/1.11.x.
The below text is copied from the body of the original PR.
This PR introduces a new model for running blackbox tests with Packer.
By blackbox testing we mean invoking the executable in a realistic but controlled environment, so we can run tests based on a user's perspective, and run shell-like pipelines on the command's outputs. This allows us to ensure Packer behaves consistently between versions, and makes sure if there's a change in a test, that it is covered and conscious.
Since we're changing how plugins are loaded with Packer 1.11, the tests included in this branch are essentially covering those changes, with eventually more and more tests that can/will be migrated to this architecture.
Overview of commits
- 1f4f2aebca2f0d27959cc5c6b13b0df8b53f3136 - ef50474a9f60acb77ee682b5dcf16371b0bb7db4 - b62e8cef9c34c755f280ba958eee409c56fa17e9 - 045b7b736555aa5b27ad841042cc8fb97ff56a76 - 56c2b5ad8b1fe4be98889adb7a92f8cf6f3dd35a - 0e2695d3e2e1d6eedb74dcea951cef30bc47691f - 316f890fe9d788d0f6f03a55635ce6567f8ec3da - 6ab7f670ad54a75af193fc287c0971b7d5c14b02 - d1b431ae270b05e8e2dce99844902f4bcbc50445 - fbb8cac6986c1c08915416a32f2d11083403dabe - 03bb62cd992352410bcdc884a87077a42b8a1188 - f8749afa16f49eb885951bb3c7727438c61e35e3 - 430f2d15cd08e70dbfa550b9a60c3b3d6e4d917b - 848188d705c330f9badf3e7a36b20b8b97869cfc - aa455014f5027c183b57edb76ad6b297a1e98700 - a62dab8ae7b247b5131cc7cf771c680dcd42b8a0 - 464dbd34dddbd94f8f7dfe2fc702875b4bfb8158 - e7e8ad07d49bdef9608db85517fa63256540f129 - c1a0002a10718fbc3f573d65f1013311044a0685 - 1b643125c0e1cb608912b7b10ab16438f7c48d91 - 96892fbca01af88f89860fb264e2ca89a1dd3527 - 4d6d3cc81e27c9253dc60d2c30ffd0a681afcbd6 - c3e4c5cde43df9c485925d45322a87c8da6c6c2e - bd79b37aa7ffd8f0dccdfe44b620a80b43c63cd3 - b31b8d30676954c0aca07c302fdac2d6bc3ea976 - b41c052c84021e4ebbf5a443bccbbc299b1ec2b4 - 0a5bdb5b7884884b353668b9915fab8fc38f0cd5 - 0c2cfe3745f1090799b68c1154b6cbdb0503c011 - d15e32cacb1c9c2998e58731d5799f1db1caabad - 7aa9bac2560aed9e2e077cb27c14f1d8d28be058 - c69ab08f9a688555d5148a056688e3a0379f1d07 - f00a49ddc9d399f067b4d7a7b28ac81b0d1d8da3 - 91eaf933754e8095cae65869c6399ac3dc51bd32 - 6f93eec99ba558eab37c2f2ef68b5aa6c4eeaf5a - 1e126da9162631102834208438f3ba94b4926a55 - 299a5af5c517af15a4806ccd15111e035cacd1c6 - 9c3f45f574b09af259fddbb4b8dbb20b64a08de8 - 7fbef0f5a71397874f184c822e1326e14a518bf6 - 124a8d8d5de132745e1ad889c29a33780955898e - 2d293e6504be83a9115513a7c14045c776216cc9 - 28bf7848692b893087af31d1dd1d2cbc7de08564 - 032114b3fce08fcbfdaeb6ee8d6f50e34400a14c - 8724e5a422ced04fa80abbfc50c7f67f7a1efa8d - 5ee3044402bdfe1a5b793a1bdbf6c384f07ff94b - a701c3a14b4ae688df32b192739564b85084532d - 99248cdcc27d5f715ec641717a9ab12c5c39b06e - 22711d176e2c7f3e86e43592d92af68b5fe50425 - 706a59ecaf54b7bf939c4abc5827bb43903627d7 - 0f0c0596318a4fd14a4521c3b352bba40f5eca1a - c856674a3ed3cb29269a8e172c90378657cf6add - 2bdf0066384c66cd422ea01edbe18b4c155fad2a - f5ca6f0c4fda18091832b105cb1ecbf699ce47fd - a976197bbf888fba2ded3833fd4deb0734ea7399 - 02da4b28e034a490b11e40e5b968055c4faafc6d - 76dbc54dc84d56b2fd336a2f2c63ca20d4fb424c - d223d2117f8796da7dbbdef5010e4c658109e3ad - 60d46ec14643a93c9850eca3f862151cc6828772 - 40a23400e9ec0c92c73797943200c3883e5609e4 - a2431ce651bc4aa600460a091520621817751161 - cde531107d953176ede5e28c71399a1d2dab95c7 - 6bffafaec58a79c6f02c626d17145ad0e7278840 - 6f299f5228ca4714a993d72500e02f87f90e148a - 57d965411fe80b328523f46c6c480e61b18cf3fa - 1345183a3093f2bbfb2176bb43d1fbcc5aa55bb3 - a693c683bcaa3581e053d225117d831dc5f770c9 - 164f091f1069688002b43c323c9ce62b14fd1780 - 37b33c53269b9087053ecd7f0484d3a04e018a05 - 59bff3bdbe910907a5b9d6f0f09ad35c19d7655c