Conjur Puppet module has a plan for an automated Windows integration test matrix

[izgeri]

This card is to create an implementation plan for adding automated tests for Windows. This should include:

• Automated test flows for Windows Server 2016 and 2019
• Manual test flows for Windows Server 2012 R2, that can be run before releases. Instructions for running them should be added to CONTRIBUTING.md

AC:

• [ ] There is a plan for adding Windows integration tests to this project added as a comment in this issue
• [ ] All related issues have been filed with estimates and put in the Puppet epic / milestone 3
• [ ] Once the issues have been filed, the estimate for this card should be moved to 0 and this issue can be closed.

Helpful resources:

• https://github.com/cyberark/conjur-puppet/tree/27-integration-tests branch, which contains some old terraform-based flows for deploying Windows / tests on AWS
• https://github.com/cyberark/conjur-puppet/issues/96 describes a way to run automated tests with Windows leveraging Windows containers
• https://github.com/cyberark/conjur-puppet/issues/89 has the manual test plan for Windows support, and at least this set of use cases should be covered / tested by the tests that will be written in this effort.

[izgeri]

See also #125

[diverdane]

Here's a gist describing the 3 methods that I was exploring for doing Windows Puppet agent integration tests: https://gist.github.com/diverdane/b4257c49890f500551fd5c0322bfbf72 I think I'm close to getting a PoC working for the 3rd method, but this will be SLOW, esp. compared to Windows containers. (It might be helpful in testing Windows 2012, though).

[izgeri]

More detailed notes from @diverdane and @doodlesbykumbi are below.

I'd still like to end this card with a recommended implementation and rough plan (aka bullets) for how to get it implemented - any thoughts on how to move forward?

Dane's notes

On the Vagrant/GCP front, I had tried 3 approaches and got stuck on all 3: https://gist.github.com/diverdane/b4257c49890f500551fd5c0322bfbf72

1. Using vagrant-google plugin, running vagrant on Jenkins server
2. Using docker-machine running on Jenkins server
3. Nested VMs: Run VirtualBox remotely on a GCP/GCE Debian VM, and use that to spin up a nested Windows VM

In all three cases, the VM gets created, but the host is unable to make initial connection with the guest VM. I think the 3rd approach is the closest to working... it might just need the right combo of WinRM configuration, but this would take more wireshark/tshark digging. (edited)

Kumbi's notes

I have a brain dump gist at https://gist.github.com/doodlesbykumbi/e392b6c0f0702f8354cf1a7dc0a33dbf for running a puppet agent on windows containers leveraging ssh port-forwarding. I think it was highlighted before that the executors can only have one job running and so port conflicts turned out to not be an issue

A windows VM with containers and ssh access works

[doodlesbykumbi]

At this point time I think there are 2 approaches available

1. Run a Windows VM (probably EC2) Jenkins agent with SSH access. This would require wrapping a windows node around a linux node in the Jenkinsfile. Then the pipelines run on the linux node as usual. To get access to the Docker daemon on the windows node and to make local ports available to containers on the windows node would require SSH port-forward, see this gist on how to do that.

2. Run a linux VM with nested virtualisation. This would require starting up a windows VM on every pipeline run (which might be slow) but would not require any fancy port-forwarding stuff because the windows VM would be on the machine running the pipeline.

[sgnn7]

The plan has been chosen since we have a PR open at https://github.com/cyberark/conjur-puppet/pull/204. Closing as completed.