approvals / ApprovalTests.cpp

Native ApprovalTests for C++ on Linux, Mac and Windows
https://approvaltestscpp.readthedocs.io/en/latest/
Apache License 2.0
310 stars 50 forks source link

useApprovalsSubdirectory() doesn't create the output directory in the expected location #196

Open Bktero opened 2 years ago

Bktero commented 2 years ago

Hello

From documentation, we can read:

Note that the sub-directory is created automatically and that it will be inside the directory containing the source code of the test, not the current working directory of the test process.

This is not what I get in my project:

image

We can see in the screenshot that the output_test_case is created in CMAKE_BINARY_DIR instead of CMAKE_SOURCE_DIR, as we would expect because this is where tests.cpp is located. I have the same issue with Makefile and Ninja generators, btw.

Best regards Pierre

claremacrae commented 2 years ago

Hi Pete, your screenshot is very clear in showing the problem.

I just checked and our GitHub Actions builds are still running and passing on a wide range of environments. The fact that they are passing tells me that the approved files in the repo (and not the build dir) are being found OK.

I think if this was a common problem, we would have had a lot more reports about it.

However, in case it's caused by a newer version of some tool or other, I would like to try and get to the bottom of it.

So the thing we need to work out is how your build differs from those that successfully place the approved and received files alongside the source code.

Is your project on GitHub, for me to test it out?

If not, please give me as much info as possible about the setup, including:

Thanks in advance.

claremacrae commented 2 years ago

I see you also submitted #195 - sorry not to have looked at that.

Is this the same problem, or something different?

Bktero commented 2 years ago

There is now a project on GitHub to reproduce the issue (at least on my computer): https://github.com/Bktero/useApprovalsSubdirectoryBug

Writing this project, I found out that the issue depends on the generator. With MinGW Makefiles, it works; with Unix Makefiles, it doesn't. I have added the presets in the project.

I thought it was a different issue from #195 but now I think #195 is not completely relevant. Indeed, with the project I have just shared, you can reproduce #195 (and something very similar). To do so, remove the call to useApprovalsSubdirectory() in test.cpp. With the "failing" preset and you will get:

C:\......\useApprovalsSubdirectoryBug\test.cpp:4: Failure:
due to unexpected exception with message:
  Failed Approval: 
Approval File Not Found 
File: "C:\......\useApprovalsSubdirectoryBug\test.MyTestCase.approved.txt"

The "successful" preset correctly creates the file. I hence tend to believe both #195 and #196 are linked (maybe the same).


Details about my setup:


Note that the project generates warnings:

C:/..../ApprovalTests.v.10.12.2.hpp:393:13: warning: 'errno_t getenv_s(size_t, char, size_t, const char*)' redeclared without dllimport attribute: previous dllimport ignored [-Wattributes]

Is this is an issue in ApprovalTests?