Test Failures on Fedora Linux

[bkmgit]

When building on Fedora linux, the following samples and tests fail:

Failed samples: Source/Samples/DOMRuler/layout_html/layout_html Failed tests: Source/test/dvobjs/test_dvobjs_system Source/test/dvobjs/test_dvobjs_logical Source/test/dvobjs/test_dvobjs_stream Source/test/extdvobjs/test_extdvobjs_math Source/test/extdvobjs/test_extdvobjs_fs Source/test/interpreter/test_vcm_eval

Full log https://download.copr.fedorainfracloud.org/results/fed500/PurC/fedora-rawhide-x86_64/05079567-PurC/builder-live.log

[VincentWei]

Obviously, all failed tests are related to DVObjs. So I think that you might not run the run_all_tests.sh script under the building directory. It is important that run this script under your building directory, or the tests can not load the test files or the external shared libraries correctly.

Please check this.

Thanks a lot.

[VincentWei]

I checked the log and found the following issues:

1. The directory to run the tests seems ok.
2. Failed samples should be ignore, because we need pass some arguments to run the samples. They are not a part of the automatic test.
3. Some tests need some prerequisites, for example, /usr/bin/bc and /usr/bin/vi.
4. It is strange that $SYS.uname failed.

I think it is not good to run the tests when packing the package, because they depend on a complete runtime environment.

[bkmgit]

Am checking on the tests, but could get the sample to pass. Made modifications to the sample script. a) Would you consider enabling GitHub actions (or similar services on Gitee/GitLab etc) to run checks before each merge? If so can add a script to do this. b) Do you wish to use a bash script for the tests, or would migrating to ctest be something worth considering?

[VincentWei]

a) Would you consider enabling GitHub actions (or similar services on Gitee/GitLab etc) to run checks before each merge? If so can add a script to do this.

This is a good idea. However, GitHub is not our working repo. Because of GFW, we use our own GitLab server, and push the repo to GitHub regularly. I think when there are more pull requests on GitHub, we will consider to enable GitHub Actions.

b) Do you wish to use a bash script for the tests, or would migrating to ctest be something worth considering?

Currently, we are using gtest. I think there is no need to migrate to ctest. But we will improve the script to run the tests according to the issues which are found by you or others.

Thanks a lot!

[bkmgit]

老师，将在Fedora中使用补丁进行测试，并禁用无法修复的失败测试自动运行。 也许吧Gitee.com 语言支持更容易！ 谢谢

[VincentWei]

In version 0.9.5, we have fixed this issue by checking the availability of the command line programs which are needed by the test cases.

Please check this. Thanks in advance.

[bkmgit]

Thanks. Will check this week.

[bkmgit]

Failed samples: Source/Samples/DOMRuler/layout_html/layout_html Can run this with specified input

Failed tests: Source/test/dvobjs/test_dvobjs_system Crashed tests: Source/test/dvobjs/test_dvobjs_datetime

Other tests pass. Thanks for the updates.

Full log https://download.copr.fedorainfracloud.org/results/fed500/PurC/fedora-rawhide-x86_64/05208980-PurC/builder-live.log.gz

[ninexue]

Please upload the log file /var/tmp/purc-tests.log, it can help us to solve the problem.

Thanks a lot!

[bkmgit]

For commit a5c4cf6, following tests fail:

/home/PurC/Source/test/dvobjs/test_dvobjs_system.cpp:978: Failure
Expected equality of these values:
  purc_variant_is_equal_to(result, expected)
    Which is: false
  true
[  FAILED  ] dvobjs.timezone (1 ms)
[ RUN      ] dvobjs.random
/home/PurC/Source/test/dvobjs/test_dvobjs_system.cpp:1100: Failure
Expected equality of these values:
  ret
    Which is: 6
  PURC_ERROR_OK
    Which is: 0
[  FAILED  ] dvobjs.random (1 ms)
[ RUN      ] dvobjs.cwd
/home/PurC/Source/test/dvobjs/test_dvobjs_system.cpp:1240: Failure
Expected equality of these values:
  ret
    Which is: 6
  PURC_ERROR_OK
    Which is: 0
[  FAILED  ] dvobjs.cwd (0 ms)
[ RUN      ] dvobjs.env
/home/PurC/Source/test/dvobjs/test_dvobjs_system.cpp:1397: Failure
Expected equality of these values:
  ret
    Which is: 6
  PURC_ERROR_OK
    Which is: 0
[  FAILED  ] dvobjs.env (0 ms)
[----------] 11 tests from dvobjs (195 ms total)

everything else passes

[bkmgit]

purc-tests.log on commit https://github.com/HVML/PurC/commit/a5c4cf631958598c236f05123e51eb4f42d21a06 purc-tests.log

[bkmgit]

From the log, maybe parsing errors?

Also an error evaluating the timezone. Trying the following program:

#include <stdio.h>
#include <stdlib.h>

int main () {
   printf("TZ   : %s\n", getenv("TZ"));
   return(0);
}

returns TZ : (null) when run without setting the time zone. TZ variable does not seem to be set by default on Fedora. Thus https://github.com/HVML/PurC/blob/master/Source/test/dvobjs/test_dvobjs_system.cpp#L889 is used and seems to return correct output in the log file. However https://github.com/HVML/PurC/blob/master/Source/PurC/dvobjs/system.c#L1293 seems to fail.

[VincentWei]

@bkmgit Thank you very much!

Your information will help us to fix the bugs.

[bkmgit]

On Fedora, PURC_SYS_TZ_DIR will have values such as /usr/share/zoneinfo/ and path would have a value such as ../usr/share/zoneinfo/Asia/Beijing so string comparisons in https://github.com/HVML/PurC/blob/master/Source/PurC/dvobjs/system.c#L1295 and https://github.com/HVML/PurC/blob/master/Source/test/dvobjs/test_dvobjs_system.cpp#L891 will fail. The pointer timezone is still returned in the test code https://github.com/HVML/PurC/blob/master/Source/test/dvobjs/test_dvobjs_system.cpp#L900 giving a memory leak which can be found using Valgrind. This causes the random, cwd and env tests to also fail.