Tests fail?

[doronbehar]

I get these weird errors when I build the package and test it:

``` Running tests... /nix/store/mnq9fvg9yqan4g1pgis01l0mqfkihpqs-cmake-3.21.2/bin/ctest --force-new-ctest-process Test project /build/source/build Start 1: YMP_NNTest.construction Start 2: YMP_NNTest.is_zero Start 3: YMP_NNTest.comparison Start 4: YMP_NNTest.parity Start 5: YMP_NNTest.to_string Start 6: YMP_NNTest.io Start 7: YMP_NNTest.shift Start 8: YMP_NNTest.sqr 1/82 Test #1: YMP_NNTest.construction .......... Passed 0.00 sec Start 9: YMP_NNTest.mul 2/82 Test #2: YMP_NNTest.is_zero ............... Passed 0.00 sec Start 10: YMP_NNTest.pow 3/82 Test #3: YMP_NNTest.comparison ............ Passed 0.00 sec Start 11: YMP_NNTest.div_rem 4/82 Test #4: YMP_NNTest.parity ................ Passed 0.00 sec Start 12: YMP_NNTest.bitwise 5/82 Test #5: YMP_NNTest.to_string ............. Passed 0.00 sec Start 13: YMP_NNTest.no_digits 6/82 Test #6: YMP_NNTest.io .................... Passed 0.00 sec Start 14: YMP_NNTest.gcd 7/82 Test #7: YMP_NNTest.shift ................. Passed 0.01 sec Start 15: YMP_ZZTest.construction 8/82 Test #8: YMP_NNTest.sqr ................... Passed 0.01 sec Start 16: YMP_ZZTest.is_zero 9/82 Test #9: YMP_NNTest.mul ................... Passed 0.01 sec Start 17: YMP_ZZTest.comparison 10/82 Test #10: YMP_NNTest.pow ................... Passed 0.01 sec Start 18: YMP_ZZTest.parity 11/82 Test #11: YMP_NNTest.div_rem ............... Passed 0.01 sec Start 19: YMP_ZZTest.to_string 12/82 Test #12: YMP_NNTest.bitwise ............... Passed 0.01 sec Start 20: YMP_ZZTest.io 13/82 Test #13: YMP_NNTest.no_digits ............. Passed 0.01 sec Start 21: YMP_ZZTest.shift 14/82 Test #14: YMP_NNTest.gcd ................... Passed 0.01 sec Start 22: YMP_ZZTest.sqr 15/82 Test #15: YMP_ZZTest.construction .......... Passed 0.01 sec Start 23: YMP_ZZTest.add 16/82 Test #16: YMP_ZZTest.is_zero ............... Passed 0.01 sec Start 24: YMP_ZZTest.mul 17/82 Test #17: YMP_ZZTest.comparison ............ Passed 0.01 sec Start 25: YMP_ZZTest.pow 18/82 Test #18: YMP_ZZTest.parity ................ Passed 0.01 sec Start 26: YMP_ZZTest.div 19/82 Test #19: YMP_ZZTest.to_string ............. Passed 0.01 sec Start 27: YMP_ZZTest.rem 20/82 Test #20: YMP_ZZTest.io .................... Passed 0.01 sec Start 28: YMP_ZZTest.no_digits 21/82 Test #21: YMP_ZZTest.shift ................. Passed 0.01 sec Start 29: YMP_ZZTest.gcd 22/82 Test #22: YMP_ZZTest.sqr ................... Passed 0.01 sec Start 30: cyacas-arithmetic.yts Process not started /build/source/tests/test-yacas [no such file or directory] 23/82 Test #30: cyacas-arithmetic.yts ............***Not Run 0.00 sec Start 31: cyacas-association.yts Process not started /build/source/tests/test-yacas [no such file or directory] 24/82 Test #31: cyacas-association.yts ...........***Not Run 0.00 sec Start 32: cyacas-binaryfactors.yts Process not started /build/source/tests/test-yacas [no such file or directory] 25/82 Test #32: cyacas-binaryfactors.yts .........***Not Run 0.00 sec Start 33: cyacas-bitops.yts Process not started /build/source/tests/test-yacas [no such file or directory] 26/82 Test #33: cyacas-bitops.yts ................***Not Run 0.00 sec Start 34: cyacas-c_tex_form.yts Process not started /build/source/tests/test-yacas [no such file or directory] 27/82 Test #34: cyacas-c_tex_form.yts ............***Not Run 0.00 sec Start 35: cyacas-calculus.yts Process not started /build/source/tests/test-yacas [no such file or directory] 28/82 Test #35: cyacas-calculus.yts ..............***Not Run 0.00 sec Start 36: cyacas-calendar.yts Process not started /build/source/tests/test-yacas [no such file or directory] 29/82 Test #36: cyacas-calendar.yts ..............***Not Run 0.00 sec Start 37: cyacas-canprove.yts Process not started /build/source/tests/test-yacas [no such file or directory] 30/82 Test #37: cyacas-canprove.yts ..............***Not Run 0.00 sec Start 38: cyacas-comments.yts Process not started /build/source/tests/test-yacas [no such file or directory] 31/82 Test #38: cyacas-comments.yts ..............***Not Run 0.00 sec Start 39: cyacas-complex.yts Process not started /build/source/tests/test-yacas [no such file or directory] 32/82 Test #39: cyacas-complex.yts ...............***Not Run 0.00 sec Start 40: cyacas-cyclotomic.yts Process not started /build/source/tests/test-yacas [no such file or directory] 33/82 Test #40: cyacas-cyclotomic.yts ............***Not Run 0.00 sec Start 41: cyacas-deriv.yts Process not started /build/source/tests/test-yacas [no such file or directory] 34/82 Test #41: cyacas-deriv.yts .................***Not Run 0.00 sec Start 42: cyacas-dimensions.yts Process not started /build/source/tests/test-yacas [no such file or directory] 35/82 Test #42: cyacas-dimensions.yts ............***Not Run 0.00 sec Start 43: cyacas-dot.yts Process not started /build/source/tests/test-yacas [no such file or directory] 36/82 Test #43: cyacas-dot.yts ...................***Not Run 0.00 sec Start 44: cyacas-GaussianIntegers.yts Process not started /build/source/tests/test-yacas [no such file or directory] 37/82 Test #44: cyacas-GaussianIntegers.yts ......***Not Run 0.00 sec Start 45: cyacas-graphs.yts Process not started /build/source/tests/test-yacas [no such file or directory] 38/82 Test #45: cyacas-graphs.yts ................***Not Run 0.00 sec Start 46: cyacas-integrate.yts Process not started /build/source/tests/test-yacas [no such file or directory] 39/82 Test #46: cyacas-integrate.yts .............***Not Run 0.00 sec Start 47: cyacas-io.yts Process not started /build/source/tests/test-yacas [no such file or directory] 40/82 Test #47: cyacas-io.yts ....................***Not Run 0.00 sec Start 48: cyacas-journal.yts Process not started /build/source/tests/test-yacas [no such file or directory] 41/82 Test #48: cyacas-journal.yts ...............***Not Run 0.00 sec Start 49: cyacas-linalg.yts Process not started /build/source/tests/test-yacas [no such file or directory] 42/82 Test #49: cyacas-linalg.yts ................***Not Run 0.00 sec Start 50: cyacas-lists.yts Process not started /build/source/tests/test-yacas [no such file or directory] 43/82 Test #50: cyacas-lists.yts .................***Not Run 0.00 sec Start 51: cyacas-logic_simplify_test.yts Process not started /build/source/tests/test-yacas [no such file or directory] 44/82 Test #51: cyacas-logic_simplify_test.yts ...***Not Run 0.00 sec Start 52: cyacas-macro.yts Process not started /build/source/tests/test-yacas [no such file or directory] 45/82 Test #52: cyacas-macro.yts .................***Not Run 0.00 sec Start 53: cyacas-matrixpower.yts Process not started /build/source/tests/test-yacas [no such file or directory] 46/82 Test #53: cyacas-matrixpower.yts ...........***Not Run 0.00 sec Start 54: cyacas-multivar.yts Process not started /build/source/tests/test-yacas [no such file or directory] 47/82 Test #54: cyacas-multivar.yts ..............***Not Run 0.00 sec Start 55: cyacas-newly.yts Process not started /build/source/tests/test-yacas [no such file or directory] 48/82 Test #55: cyacas-newly.yts .................***Not Run 0.00 sec Start 56: cyacas-nthroot.yts Process not started /build/source/tests/test-yacas [no such file or directory] 49/82 Test #56: cyacas-nthroot.yts ...............***Not Run 0.00 sec Start 57: cyacas-numbers.yts Process not started /build/source/tests/test-yacas [no such file or directory] 50/82 Test #57: cyacas-numbers.yts ...............***Not Run 0.00 sec Start 58: cyacas-numerics.yts Process not started /build/source/tests/test-yacas [no such file or directory] 51/82 Test #58: cyacas-numerics.yts ..............***Not Run 0.00 sec Start 59: cyacas-nummethods.yts Process not started /build/source/tests/test-yacas [no such file or directory] 52/82 Test #59: cyacas-nummethods.yts ............***Not Run 0.00 sec Start 60: cyacas-ode.yts Process not started /build/source/tests/test-yacas [no such file or directory] 53/82 Test #60: cyacas-ode.yts ...................***Not Run 0.00 sec Start 61: cyacas-openmath.yts Process not started /build/source/tests/test-yacas [no such file or directory] 54/82 Test #61: cyacas-openmath.yts ..............***Not Run 0.00 sec Start 62: cyacas-orthopoly.yts Process not started /build/source/tests/test-yacas [no such file or directory] 55/82 Test #62: cyacas-orthopoly.yts .............***Not Run 0.00 sec Start 63: cyacas-outer.yts Process not started /build/source/tests/test-yacas [no such file or directory] 56/82 Test #63: cyacas-outer.yts .................***Not Run 0.00 sec Start 64: cyacas-padic.yts Process not started /build/source/tests/test-yacas [no such file or directory] 57/82 Test #64: cyacas-padic.yts .................***Not Run 0.00 sec Start 65: cyacas-physics.yts Process not started /build/source/tests/test-yacas [no such file or directory] 58/82 Test #65: cyacas-physics.yts ...............***Not Run 0.00 sec Start 66: cyacas-plots.yts Process not started /build/source/tests/test-yacas [no such file or directory] 59/82 Test #66: cyacas-plots.yts .................***Not Run 0.00 sec Start 67: cyacas-poly.yts Process not started /build/source/tests/test-yacas [no such file or directory] 60/82 Test #67: cyacas-poly.yts ..................***Not Run 0.00 sec Start 68: cyacas-predicates.yts Process not started /build/source/tests/test-yacas [no such file or directory] 61/82 Test #68: cyacas-predicates.yts ............***Not Run 0.00 sec Start 69: cyacas-products.yts Process not started /build/source/tests/test-yacas [no such file or directory] 62/82 Test #69: cyacas-products.yts ..............***Not Run 0.00 sec Start 70: cyacas-programming.yts Process not started /build/source/tests/test-yacas [no such file or directory] 63/82 Test #70: cyacas-programming.yts ...........***Not Run 0.00 sec Start 71: cyacas-radsimp.yts Process not started /build/source/tests/test-yacas [no such file or directory] 64/82 Test #71: cyacas-radsimp.yts ...............***Not Run 0.00 sec Start 72: cyacas-rational.yts Process not started /build/source/tests/test-yacas [no such file or directory] 65/82 Test #72: cyacas-rational.yts ..............***Not Run 0.00 sec Start 73: cyacas-regress.yts Process not started /build/source/tests/test-yacas [no such file or directory] 66/82 Test #73: cyacas-regress.yts ...............***Not Run 0.00 sec Start 74: cyacas-scopestack.yts Process not started /build/source/tests/test-yacas [no such file or directory] 67/82 Test #74: cyacas-scopestack.yts ............***Not Run 0.00 sec Start 75: cyacas-simplify.yts Process not started /build/source/tests/test-yacas [no such file or directory] 68/82 Test #75: cyacas-simplify.yts ..............***Not Run 0.00 sec Start 76: cyacas-solve.yts Process not started /build/source/tests/test-yacas [no such file or directory] 69/82 Test #76: cyacas-solve.yts .................***Not Run 0.00 sec Start 77: cyacas-sturm.yts Process not started /build/source/tests/test-yacas [no such file or directory] 70/82 Test #77: cyacas-sturm.yts .................***Not Run 0.00 sec Start 78: cyacas-sums.yts Process not started /build/source/tests/test-yacas [no such file or directory] 71/82 Test #78: cyacas-sums.yts ..................***Not Run 0.00 sec Start 79: cyacas-tensors.yts Process not started /build/source/tests/test-yacas [no such file or directory] 72/82 Test #79: cyacas-tensors.yts ...............***Not Run 0.00 sec Start 80: cyacas-tr.yts Process not started /build/source/tests/test-yacas [no such file or directory] 73/82 Test #80: cyacas-tr.yts ....................***Not Run 0.00 sec Start 81: cyacas-trace.yts Process not started /build/source/tests/test-yacas [no such file or directory] 74/82 Test #81: cyacas-trace.yts .................***Not Run 0.00 sec Start 82: cyacas-transforms.yts Process not started /build/source/tests/test-yacas [no such file or directory] 75/82 Test #82: cyacas-transforms.yts ............***Not Run 0.00 sec 76/82 Test #23: YMP_ZZTest.add ................... Passed 0.03 sec 77/82 Test #24: YMP_ZZTest.mul ................... Passed 0.03 sec 78/82 Test #25: YMP_ZZTest.pow ................... Passed 0.03 sec 79/82 Test #26: YMP_ZZTest.div ................... Passed 0.03 sec 80/82 Test #27: YMP_ZZTest.rem ................... Passed 0.03 sec 81/82 Test #28: YMP_ZZTest.no_digits ............. Passed 0.03 sec 82/82 Test #29: YMP_ZZTest.gcd ................... Passed 0.03 sec 35% tests passed, 53 tests failed out of 82 Total Test time (real) = 0.05 sec The following tests FAILED: 30 - cyacas-arithmetic.yts (BAD_COMMAND) 31 - cyacas-association.yts (BAD_COMMAND) 32 - cyacas-binaryfactors.yts (BAD_COMMAND) 33 - cyacas-bitops.yts (BAD_COMMAND) 34 - cyacas-c_tex_form.yts (BAD_COMMAND) 35 - cyacas-calculus.yts (BAD_COMMAND) 36 - cyacas-calendar.yts (BAD_COMMAND) 37 - cyacas-canprove.yts (BAD_COMMAND) 38 - cyacas-comments.yts (BAD_COMMAND) 39 - cyacas-complex.yts (BAD_COMMAND) 40 - cyacas-cyclotomic.yts (BAD_COMMAND) 41 - cyacas-deriv.yts (BAD_COMMAND) 42 - cyacas-dimensions.yts (BAD_COMMAND) 43 - cyacas-dot.yts (BAD_COMMAND) 44 - cyacas-GaussianIntegers.yts (BAD_COMMAND) 45 - cyacas-graphs.yts (BAD_COMMAND) 46 - cyacas-integrate.yts (BAD_COMMAND) 47 - cyacas-io.yts (BAD_COMMAND) 48 - cyacas-journal.yts (BAD_COMMAND) 49 - cyacas-linalg.yts (BAD_COMMAND) 50 - cyacas-lists.yts (BAD_COMMAND) 51 - cyacas-logic_simplify_test.yts (BAD_COMMAND) 52 - cyacas-macro.yts (BAD_COMMAND) 53 - cyacas-matrixpower.yts (BAD_COMMAND) 54 - cyacas-multivar.yts (BAD_COMMAND) 55 - cyacas-newly.yts (BAD_COMMAND) 56 - cyacas-nthroot.yts (BAD_COMMAND) 57 - cyacas-numbers.yts (BAD_COMMAND) 58 - cyacas-numerics.yts (BAD_COMMAND) 59 - cyacas-nummethods.yts (BAD_COMMAND) 60 - cyacas-ode.yts (BAD_COMMAND) 61 - cyacas-openmath.yts (BAD_COMMAND) 62 - cyacas-orthopoly.yts (BAD_COMMAND) 63 - cyacas-outer.yts (BAD_COMMAND) 64 - cyacas-padic.yts (BAD_COMMAND) 65 - cyacas-physics.yts (BAD_COMMAND) 66 - cyacas-plots.yts (BAD_COMMAND) 67 - cyacas-poly.yts (BAD_COMMAND) 68 - cyacas-predicates.yts (BAD_COMMAND) 69 - cyacas-products.yts (BAD_COMMAND) 70 - cyacas-programming.yts (BAD_COMMAND) 71 - cyacas-radsimp.yts (BAD_COMMAND) 72 - cyacas-rational.yts (BAD_COMMAND) 73 - cyacas-regress.yts (BAD_COMMAND) 74 - cyacas-scopestack.yts (BAD_COMMAND) 75 - cyacas-simplify.yts (BAD_COMMAND) 76 - cyacas-solve.yts (BAD_COMMAND) 77 - cyacas-sturm.yts (BAD_COMMAND) 78 - cyacas-sums.yts (BAD_COMMAND) 79 - cyacas-tensors.yts (BAD_COMMAND) 80 - cyacas-tr.yts (BAD_COMMAND) 81 - cyacas-trace.yts (BAD_COMMAND) 82 - cyacas-transforms.yts (BAD_COMMAND) ```

[grzegorzmazur]

The yacas tests fall into two categories: standard unit tests, which for now only cover some of the big integer operations, and integration tests which use a script (test-yacas) to run yacas, perform some calculations and check if the results are correct. The log you've submitted shows that the unit tests run just fine, but the driver for integration tests (test-yacas) cannot be found.

It looks weird, because it should be there and it seems to work not only on my machine but also during the Continuous Integration (CI) tests.

OK, could you please check if the test-yacas script is there at all?

[doronbehar]

OK, could you please check if the test-yacas script is there at all?

Yes it's there. I think what it doesn't find is the files with the prefix cyacas- and the suffix .yts like cyacas-integrate.yts. They are named without the cyacas- prefix in the tests directory.

[grzegorzmazur]

These are names of the tests, not the files. The prefix is to differentiate between tests of c++ and java implementations of the engine; the scripts and tests files are identical for both. But OK, could you please try to run

ctest --verbose

in the build directory?

[doronbehar]

I've found 1 issue - the test-yacas script had the shebang #!/bin/bash and in NixOS' build sandbox that file doesn't exist, but Nix is familiar with this common issue and I've overcome that.

Now I see that ctest still fails, with a different error. E.g:

yacas> Command = "'/build/source/build/cyacas/yacas/yacas' -pc --rootdir /build/source/scripts"
yacas> Version =
yacas> Date = "2021-11-30 17:09:11"
yacas> Logfile = "yacas-logfile.txt"
yacas> /*===================
yacas> Failed =  transforms.yts
yacas>  1 of 1 tests FAILED
yacas> ===================*/
yacas> 82/82 Test #81: cyacas-trace.yts .................***Failed    0.08 sec
yacas> /build/source/tests/test-yacas: line 43: '/build/source/build/cyacas/yacas/yacas': No such file or directory
yacas> /build/source/tests/test-yacas: line 44: '/build/source/build/cyacas/yacas/yacas': No such file or directory

And indeed running /build/source/build/cyacas/yacas/yacas, results in a segmentation fault:

[nixbld@localhost:/var/lib/cntr/build/source/build/cyacas/yacas]$ ./yacas -pc --rootdir ../../../scripts ../../../tests
Segmentation fault (core dumped)

It's peculiar because the executable seems rather functional when the build is complete after make install.

[grzegorzmazur]

Why is ctest trying to run /build/source/build/cyacas/yacas/yacas when your yacas seem to be in /var/lib/cntr/build/source/build/cyacas/yacas?

The ./yacas -pc --rootdir ../../../scripts ../../../tests most likely segfaults because it gets directory ../../../tests as a file to run. This should be obviously fixed, but to actually check if yacas executes tests correctly try for example ./yacas -pc --rootdir ../../../scripts ../../../tests/integrate.yts

[doronbehar]

Why is ctest trying to run /build/source/build/cyacas/yacas/yacas when your yacas seem to be in /var/lib/cntr/build/source/build/cyacas/yacas?

While the build is happening, this is the same directory. If I inspect what went wrong in the build after it failed, I enter a sort of container in which the build files are actually located in /var/lib/cntr/... That's a NixOS specific detail.

Running manually:

    /build/source/tests/test-yacas \
      "'/build/source/build/cyacas/yacas/yacas' -pc \
      --rootdir /build/source/scripts" \
      "/build/source/tests/integrate.yts"

Results in errors such as:

yacas> /build/source/tests/test-yacas: line 106: '/build/source/build/cyacas/yacas/yacas': No such file
or directory
yacas> /build/source/tests/test-yacas: line 107: '/build/source/build/cyacas/yacas/yacas': No such file
or directory
yacas> /build/source/tests/test-yacas: line 108: '/build/source/build/cyacas/yacas/yacas': No such file
or directory
yacas> {"Total", , "2021-11-30 17:56:17", , , , Pass},
yacas> rm: cannot remove '/tmp/test-yacas.897': No such file or directory
yacas> rm: cannot remove '/tmp/time-yacas.897': No such file or directory

But running:

./yacas -pc --rootdir ../../../scripts ../../../tests/integrate.yts

Seems to exit cleanly.

The problem seems to be lines like this:

https://github.com/grzegorzmazur/yacas/blob/afade5b432c9d602daefe58abfd88bd985d73e51/tests/test-yacas#L38-L40

Where this directory is not available in NixOS' build sandbox. We should use mktemp there. I'll try to write a patch later tomorrow.