test_api_dump_layer is failing on Windows

[lunarpapillo]

We're blocked on enabling VulkanTools testing because tests are failing on Windows: http://tcubuser.lunarg.localdomain:8080/job/VulkanTools/3270/BITS=64,BUILD_MODE=Release,label=Windows-NVIDIA-20XX-rolling-NDA/console

test 1
      Start  1: test_api_dump_layer

1: Test command: C:\j\vt\Release64\VulkanTools\build\layersvt\test\Release\test_api_dump_layer.exe
1: Working Directory: C:/j/vt/Release64/VulkanTools/build/layersvt/test
1: Environment variables: 
1:  VK_LAYER_PATH=C:/j/vt/Release64/VulkanTools/build/layersvt/Release
1: Test timeout computed to be: 10000000
1: [==========] Running 1 test from 1 test suite.
1: [----------] Global test environment set-up.
1: [----------] 1 test from ApiDumpTests
1: [ RUN      ] ApiDumpTests.init_layer
1: C:\j\vt\Release64\VulkanTools\layersvt\test\test_api_dump.cpp(55): error: Value of: file != 0
1:   Actual: false
1: Expected: true
1: 
1: [  FAILED  ] ApiDumpTests.init_layer (42 ms)
1: [----------] 1 test from ApiDumpTests (42 ms total)

[lunarpapillo]

Still failing. Try again: http://tcubuser.lunarg.localdomain:8080/job/VulkanTools/3331/BITS=64,BUILD_MODE=Debug,label=Windows-NVIDIA-20XX-rolling-NDA/console

1: [----------] 1 test from ApiDumpTests
1: [ RUN      ] ApiDumpTests.init_layer
1: C:\j\vt\Debug64\VulkanTools\layersvt\test\test_api_dump.cpp(55): error: Value of: file != 0
1:   Actual: false
1: Expected: true
1: 
1: [  FAILED  ] ApiDumpTests.init_layer (56 ms)
1: [----------] 1 test from ApiDumpTests (56 ms total)

[lunarpapillo]

Offhand, how are you testing this by hand? I'm curious why it passed in your environment but not in CI... maybe I'm doing something wrong?

[charles-lunarg]

I'm testing it by running both the exe and ctest locally. Maybe its due to how the test is being run (which may mess it up?)

[lunarpapillo]

What's your working directory?

INFO: executing #8: ctest -V --output-on-failure -C Debug
ERROR: execute() call failed after 11.11 seconds (11.11 seconds): returncode=8 (0x8); execution log index: 8, working directory: C:\j\vt\Debug64\VulkanTools\dbuild

but that's working for all the other tests, not sure why this one would be different...

[lunarpapillo]

If you want to test the way CI would with the VulkanTests change, run a VulkanTools job by hand and set:

• TEST_COMMIT: bob-tools-ctest
• EMAIL_RESULTS: anything that doesn't include my e-mail :wink:

[charles-lunarg]

The working directory is the build directory since that is where ctest expects to be run from. For just running the executable directly it was also the build directory.

I have a local run of the test script which fails like we are seeing in CI, so I think I can repro and fix it.

[charles-lunarg]

So the issue is indeed the current working directory. The tests had a hardcoded path of 'where' the output would be, which doesn't hold true for windows. Making the tests query what the current working directory is then searching there solves the issue: https://github.com/LunarG/VulkanTools/pull/2070

Now the reason the monitor and screenshot layer don't fail is because neither of them try to open any files that were written out.

And I validated the fix by running the test script locally rather than just ctest on its lonesome.

[lunarpapillo]

Verified. Merged the bob-tools-ctest branch. Testing should now be live for VulkanTools.