Closed hab6 closed 5 months ago
Hi @hab6 One of the changes in the latest release was updating an expected for a smoke test; if you update to the latest release of Tableau (from the tab query path it looks like you're using 2023.3) it should go back to passing.
You can also ignore the failing smoke test and run all tests using python -m tdvt.tdvt run {ds name} --force-run
.
If that doesn't work,
python -m tdvt.tdvt --verbose run {ds name} --verify
will produce a verbose tdvt_log_combined file for more debugging.
Sorry, that should have been --force-run
, typing too fast 😄 I also fixed it above.
Thanks for the suggestions @lukewrites
The SDK repository README states that the latest, non-beta version of Tableau Desktop/Server is 2023.3, which is the version used when I originally encountered the problem (and still do).
Per your recommendation to update to the latest release, I found and installed Tableau Desktop version 2024.1 and the smoke test still fails. Verifying the versions used for this latest test:
Tableau Desktop: ./tds/cast_calcs.mhds.tds: build 20241.24.0208.0337
Tableau Desktop: ./tds/Staples.mhds.tds: build 20241.24.0208.0337
Connector SDK: ./tdvt_log_combined.txt: INFO TDVT version: 2.13.7
Using --force-run
seems to work around the problem so that the TDVT test run ignores the failing smoke test (although the pass/fail rate is slightly worse than with SDK version 2.13.4
).
C:\tdvtTest\wksp> python -m tdvt.tdvt run mhds --force-run
.....
2 smoke test(s) ran. 0 smoke tests disabled.
Smoke tests ran in 12.69 seconds.
1 smoke test(s) failed. Please check logs for information.
.....
Test Count: 873 tests
Passed tests: 853
Failed tests: 20
Tests run: 873
Disabled tests: 0
Skipped tests: 0
The failing smoke test looks to be StaplesConnectionTestmhds, which appears to be due to trying to match strings to numbers (although the values are the same). Here is the DIFF file: actual.setup.staples.connection.test_diff.xml.txt
Actian internal tracking issue II-13736
Sorry, Michael, I was wrong in the above. Are the newly failing tests in the lod.9_Fixed as Dimension
and/or lod.calc.1.TopN
?
You can ignore the failing smoke tests since those are caused by the new expected file.
(On my system) Here are the tests that succeed with SDK 2.13.4
and fail with 2.13.7
Test Name | Test Case | Comment |
---|---|---|
staples.connection.test | logicaltests/ setup/ connection_test/ setup.staples.connection.test |
Smoke test: Succeeds with 2.13.4 Actual/Expected mismatch with 2.13.7 |
string.space.empty | SPACE(0) | New in 2.13.7 . Did not exist in 2.13.4 . |
lod.9_Fixed as Dimension | logicaltests/ setup/ lod/ setup.lod.9_Fixed as Dimension |
Succeeds with 2.13.4 Actual/Expected mismatch with 2.13.7 |
lod.calc.1.TopN | logicaltests/ setup/ lod/ setup.lod.calc.1.TopN |
Succeeds with 2.13.4 Actual/Expected mismatch with 2.13.7 |
@hab6 the test-fix
branch provides the old expecteds for the LOD tests and connection test, if you'd like to try it out. This was caused by a bug fixed in an upcoming point release. When the new release comes out I'll ping the thread and also remove the alternate expecteds.
The string.space.empty
test is new; that's a test failure to verify.
Thanks!
About You: Name: Michael Habiger Company: Actian Corporation
Your question: I am encountering a smoke test failure every time when attempting to run the TDVT tests using the latest
2.13.7
test suite.My Environment: Microsoft Windows [Version 10.0.19045.3930] Tableau Desktop 2023.3.3 (20233.24.0202.1238) Python 3.10.7
Result: When executing
python -m tdvt.tdvt run mhds
(with or without the--verify
option), the execution always reports a smoke test failure :There are no errors in tdvt_log_combined.txt and I see no relevant errors in other files/logs.
Using SDK version
2.13.4
works: Running the test suite with an exact same setup except using SDK version2.13.4
instead of2.13.7
allows the test suite to run properly.Also, if the *.tds files that fail under version
2.13.7
are moved and used unaltered in an equivalent setup with version2.13.4
, the tests run successfully with a 98% pass rate:Check of the SDK documentation Troubleshooting section: I have reviewed the Frequently found issues and troubleshooting section of the TDVT documentation.
2.13.4
which runs successfully.INFO Found tabquery at [C:\Program Files\Tableau\Tableau 2023.3\bin\tabquerytool.exe]
Files: tdvt_log_combined.txt cast_calcs.mhds.tds.txt Staples.mhds.tds.txt