[Auditbeat] Failing x-pack/auditbeat tests on Windows

[cwurm]

Flaky Test

• Test Name: process unit test, and system tests in general (both Windows only)
• Link: Unit test and system tests.
• Branch: master, 6.x, 6.6
• Artifact Link: na
• Notes: The unit test failure is caused by an error during process collection. https://github.com/elastic/beats/pull/9693 will fix the test failure by collecting the errors instead of stopping, but it will need more investigation into what is failing. The system test failure is an import error that is still a mystery to me.

Stack Trace

process unit test, e.g. here

10:29:11 >> go test: Unit Testing
10:29:18 FAILURES:
10:29:18 Error: go test failed: 1 test failures
10:29:18 Package: github.com/elastic/beats/x-pack/auditbeat/module/system/process
10:29:18 Test:    TestData
10:29:18 C:/Users/jenkins/workspace/elastic+beats+master+multijob-windows/beat/x-pack/auditbeat/label/windows/src/github.com/elastic/beats/x-pack/auditbeat/module/system/process/process.go:295
10:29:18 C:/Users/jenkins/workspace/elastic+beats+master+multijob-windows/beat/x-pack/auditbeat/label/windows/src/github.com/elastic/beats/x-pack/auditbeat/module/system/process/process.go:184
10:29:18 C:/Users/jenkins/workspace/elastic+beats+master+multijob-windows/beat/x-pack/auditbeat/label/windows/src/github.com/elastic/beats/x-pack/auditbeat/module/system/process/process.go:161
10:29:18 C:/Users/jenkins/workspace/elastic+beats+master+multijob-windows/beat/x-pack/auditbeat/label/windows/src/github.com/elastic/beats/metricbeat/mb/testing/modules.go:203
10:29:18 C:/Users/jenkins/workspace/elastic+beats+master+multijob-windows/beat/x-pack/auditbeat/label/windows/src/github.com/elastic/beats/x-pack/auditbeat/module/system/process/process_test.go:16
10:29:18 C:/Users/jenkins.BEATS-CI-WINDOW/.gvm/versions/go1.11.4.windows.amd64/src/testing/testing.go:827
10:29:18 C:/Users/jenkins.BEATS-CI-WINDOW/.gvm/versions/go1.11.4.windows.amd64/src/runtime/asm_amd64.s:1333
10:29:18 C:/Users/jenkins/workspace/elastic+beats+master+multijob-windows/beat/x-pack/auditbeat/label/windows/src/github.com/elastic/beats/x-pack/auditbeat/module/system/process/process.go:186
10:29:18 C:/Users/jenkins/workspace/elastic+beats+master+multijob-windows/beat/x-pack/auditbeat/label/windows/src/github.com/elastic/beats/x-pack/auditbeat/module/system/process/process.go:161
10:29:18 C:/Users/jenkins/workspace/elastic+beats+master+multijob-windows/beat/x-pack/auditbeat/label/windows/src/github.com/elastic/beats/metricbeat/mb/testing/modules.go:203
10:29:18 C:/Users/jenkins/workspace/elastic+beats+master+multijob-windows/beat/x-pack/auditbeat/label/windows/src/github.com/elastic/beats/x-pack/auditbeat/module/system/process/process_test.go:16
10:29:18 C:/Users/jenkins.BEATS-CI-WINDOW/.gvm/versions/go1.11.4.windows.amd64/src/testing/testing.go:827
10:29:18 C:/Users/jenkins.BEATS-CI-WINDOW/.gvm/versions/go1.11.4.windows.amd64/src/runtime/asm_amd64.s:1333
10:29:18 ----

System tests, e.g. here:

11:31:48 >> go test: Unit Test Passed
11:31:48 System testing x-pack/auditbeat
11:31:59 E
11:31:59 ======================================================================
11:31:59 ERROR: Failure: ImportError (No module named metricbeat)
11:31:59 ----------------------------------------------------------------------
11:31:59 Traceback (most recent call last):
11:31:59   File "c:\python27\lib\site-packages\nose\loader.py", line 418, in loadTestsFromName
11:31:59     addr.filename, addr.module)
11:31:59   File "c:\python27\lib\site-packages\nose\importer.py", line 47, in importFromPath
11:31:59     return self.importFromDir(dir_path, fqname)
11:31:59   File "c:\python27\lib\site-packages\nose\importer.py", line 94, in importFromDir
11:31:59     mod = load_module(part_fqname, fh, filename, desc)
11:31:59   File "C:\Users\jenkins\workspace\elastic+beats+pull-request+multijob-windows\beat\x-pack\auditbeat\label\windows\src\github.com\elastic\beats\x-pack\auditbeat\tests\system\test_metricsets.py", line 9, in <module>
11:31:59     from auditbeat_xpack import *
11:31:59   File "C:\Users\jenkins\workspace\elastic+beats+pull-request+multijob-windows\beat\x-pack\auditbeat\label\windows\src\github.com\elastic\beats\x-pack\auditbeat\tests\system\auditbeat_xpack.py", line 5, in <module>
11:31:59     from auditbeat import BaseTest as AuditbeatTest
11:31:59   File "C:\Users\jenkins\workspace\elastic+beats+pull-request+multijob-windows\beat\x-pack\auditbeat\label\windows\src\github.com\elastic\beats\x-pack\auditbeat\tests\system\../../../../auditbeat/tests/system\auditbeat.py", line 11, in <module>
11:31:59     from metricbeat import BaseTest as MetricbeatTest
11:31:59 ImportError: No module named metricbeat

[elasticmachine]

Pinging @elastic/secops

[cwurm]

The ImportError in the system test is fixed with https://github.com/elastic/beats/pull/9750.

Both unit and system test for the process metricset are skipped on Windows for now because of problems in the data collection.