search

lunarmodules / Penlight

A set of pure Lua libraries focusing on input data handling (such as reading configuration files), functional programming (such as map, reduce, placeholder expressions,etc), and OS path management. Much of the functionality is inspired by the Python standard libraries.
https://lunarmodules.github.io/Penlight/
MIT License
1.9k stars 238 forks source link

Test suite failing on Arch Linux #440

Closed alerque closed 2 years ago

alerque commented 2 years ago

I tried updating the Arch Linux package to 1.13.0 and failed, then tried checking out the repo and running the test suite locally. Both fail the same way. I see it passing in CI with similar dependency versions (Lua etc.) so I'm not sure at this point what gives but opening this to track it.

Relevant output:

$ lua -v
Lua 5.4.4  Copyright (C) 1994-2022 Lua.org, PUC-Rio

$ busted --version
2.0.0-0

$ busted spec/utils-enum_spec.lua
[==========] Running tests from scanned files.
[----------] Global test environment setup.
[----------] Running tests from spec/utils-enum_spec.lua
[ RUN      ] spec/utils-enum_spec.lua @ 13: pl.utils enum() creating accepts a vararg
[       OK ] spec/utils-enum_spec.lua @ 13: pl.utils enum() creating accepts a vararg (0.04 ms)
[ RUN      ] spec/utils-enum_spec.lua @ 23: pl.utils enum() creating vararg entries must be strings
spec/utils-enum_spec.lua:28: Expected a different error.
Caught:
(string) 'invalid value (nil) at index 2 in table for 'concat''
Expected:
(string) 'argument 2 expected a 'string', got a 'nil''

stack traceback:
    spec/utils-enum_spec.lua:28: in function <spec/utils-enum_spec.lua:23>

[  FAILED  ] spec/utils-enum_spec.lua @ 23: pl.utils enum() creating vararg entries must be strings (0.53 ms)
[ RUN      ] spec/utils-enum_spec.lua @ 34: pl.utils enum() creating vararg requires at least 1 entry
spec/utils-enum_spec.lua:35: Expected a different error.
Caught:
(string) 'argument 1 expected a 'string', got a 'nil''
Expected:
(string) 'expected at least 1 entry'

stack traceback:
    spec/utils-enum_spec.lua:35: in function <spec/utils-enum_spec.lua:34>

[  FAILED  ] spec/utils-enum_spec.lua @ 34: pl.utils enum() creating vararg requires at least 1 entry (0.50 ms)
[ RUN      ] spec/utils-enum_spec.lua @ 41: pl.utils enum() creating accepts an array
/usr/share/lua/5.4/pl/utils.lua:283: argument 1 expected a 'string', got a 'table'

stack traceback:
    /usr/share/lua/5.4/pl/utils.lua:244: in function 'pl.utils.assert_arg'
    /usr/share/lua/5.4/pl/utils.lua:283: in function 'pl.utils.enum'
    spec/utils-enum_spec.lua:42: in function <spec/utils-enum_spec.lua:41>

[  ERROR   ] spec/utils-enum_spec.lua @ 41: pl.utils enum() creating accepts an array (0.44 ms)
[ RUN      ] spec/utils-enum_spec.lua @ 51: pl.utils enum() creating array entries must be strings
spec/utils-enum_spec.lua:52: Expected a different error.
Caught:
(string) 'argument 1 expected a 'string', got a 'table''
Expected:
(string) 'expected 'string' but got 'number' at index 2'

stack traceback:
    spec/utils-enum_spec.lua:52: in function <spec/utils-enum_spec.lua:51>

[  FAILED  ] spec/utils-enum_spec.lua @ 51: pl.utils enum() creating array entries must be strings (0.51 ms)
[ RUN      ] spec/utils-enum_spec.lua @ 58: pl.utils enum() creating array requires at least 1 entry
spec/utils-enum_spec.lua:59: Expected a different error.
Caught:
(string) 'argument 1 expected a 'string', got a 'table''
Expected:
(string) 'expected at least 1 entry'

stack traceback:
    spec/utils-enum_spec.lua:59: in function <spec/utils-enum_spec.lua:58>

[  FAILED  ] spec/utils-enum_spec.lua @ 58: pl.utils enum() creating array requires at least 1 entry (0.99 ms)
[ RUN      ] spec/utils-enum_spec.lua @ 65: pl.utils enum() creating accepts a hash-table
/usr/share/lua/5.4/pl/utils.lua:283: argument 1 expected a 'string', got a 'table'

stack traceback:
    /usr/share/lua/5.4/pl/utils.lua:244: in function 'pl.utils.assert_arg'
    /usr/share/lua/5.4/pl/utils.lua:283: in function 'pl.utils.enum'
    spec/utils-enum_spec.lua:66: in function <spec/utils-enum_spec.lua:65>

[  ERROR   ] spec/utils-enum_spec.lua @ 65: pl.utils enum() creating accepts a hash-table (0.44 ms)
[ RUN      ] spec/utils-enum_spec.lua @ 77: pl.utils enum() creating hash-table keys must be strings
spec/utils-enum_spec.lua:78: Expected a different error.
Caught:
(string) 'argument 1 expected a 'string', got a 'table''
Expected:
(string) 'expected key to be 'string' but got 'table''

stack traceback:
    spec/utils-enum_spec.lua:78: in function <spec/utils-enum_spec.lua:77>

[  FAILED  ] spec/utils-enum_spec.lua @ 77: pl.utils enum() creating hash-table keys must be strings (0.50 ms)
[ RUN      ] spec/utils-enum_spec.lua @ 84: pl.utils enum() creating hash-table requires at least 1 entry
spec/utils-enum_spec.lua:85: Expected a different error.
Caught:
(string) 'argument 1 expected a 'string', got a 'table''
Expected:
(string) 'expected at least 1 entry'

stack traceback:
    spec/utils-enum_spec.lua:85: in function <spec/utils-enum_spec.lua:84>

[  FAILED  ] spec/utils-enum_spec.lua @ 84: pl.utils enum() creating hash-table requires at least 1 entry (0.51 ms)
[ RUN      ] spec/utils-enum_spec.lua @ 91: pl.utils enum() creating accepts a combined array/hash-table
/usr/share/lua/5.4/pl/utils.lua:283: argument 1 expected a 'string', got a 'table'

stack traceback:
    /usr/share/lua/5.4/pl/utils.lua:244: in function 'pl.utils.assert_arg'
    /usr/share/lua/5.4/pl/utils.lua:283: in function 'pl.utils.enum'
    spec/utils-enum_spec.lua:92: in function <spec/utils-enum_spec.lua:91>

[  ERROR   ] spec/utils-enum_spec.lua @ 91: pl.utils enum() creating accepts a combined array/hash-table (0.47 ms)
[ RUN      ] spec/utils-enum_spec.lua @ 105: pl.utils enum() creating keys must be unique with combined array/has-table
spec/utils-enum_spec.lua:106: Expected a different error.
Caught:
(string) 'argument 1 expected a 'string', got a 'table''
Expected:
(string) 'duplicate entry in array and hash part: 'FILE_NOT_FOUND''

stack traceback:
    spec/utils-enum_spec.lua:106: in function <spec/utils-enum_spec.lua:105>

[  FAILED  ] spec/utils-enum_spec.lua @ 105: pl.utils enum() creating keys must be unique with combined array/has-table (0.50 ms)
[ RUN      ] spec/utils-enum_spec.lua @ 125: pl.utils enum() accessing errors on unknown values
[       OK ] spec/utils-enum_spec.lua @ 125: pl.utils enum() accessing errors on unknown values (0.03 ms)
[ RUN      ] spec/utils-enum_spec.lua @ 132: pl.utils enum() accessing errors on setting new keys
[       OK ] spec/utils-enum_spec.lua @ 132: pl.utils enum() accessing errors on setting new keys (0.03 ms)
[ RUN      ] spec/utils-enum_spec.lua @ 139: pl.utils enum() accessing keys can have 'format' placeholders
[       OK ] spec/utils-enum_spec.lua @ 139: pl.utils enum() accessing keys can have 'format' placeholders (0.03 ms)
[ RUN      ] spec/utils-enum_spec.lua @ 157: pl.utils enum() calling returns error on unknown values
[       OK ] spec/utils-enum_spec.lua @ 157: pl.utils enum() calling returns error on unknown values (0.04 ms)
[ RUN      ] spec/utils-enum_spec.lua @ 164: pl.utils enum() calling returns value on success
[       OK ] spec/utils-enum_spec.lua @ 164: pl.utils enum() calling returns value on success (0.03 ms)
[----------] 16 tests from spec/utils-enum_spec.lua (17.32 ms total)

[----------] Global test environment teardown.
[==========] 16 tests from 1 test file ran. (17.39 ms total)
[  PASSED  ] 6 tests.
[  FAILED  ] 7 tests, listed below:
[  FAILED  ] spec/utils-enum_spec.lua @ 23: pl.utils enum() creating vararg entries must be strings
[  FAILED  ] spec/utils-enum_spec.lua @ 34: pl.utils enum() creating vararg requires at least 1 entry
[  FAILED  ] spec/utils-enum_spec.lua @ 51: pl.utils enum() creating array entries must be strings
[  FAILED  ] spec/utils-enum_spec.lua @ 58: pl.utils enum() creating array requires at least 1 entry
[  FAILED  ] spec/utils-enum_spec.lua @ 77: pl.utils enum() creating hash-table keys must be strings
[  FAILED  ] spec/utils-enum_spec.lua @ 84: pl.utils enum() creating hash-table requires at least 1 entry
[  FAILED  ] spec/utils-enum_spec.lua @ 105: pl.utils enum() creating keys must be unique with combined array/has-table
[  ERROR   ] 3 errors, listed below:
[  ERROR   ] spec/utils-enum_spec.lua @ 41: pl.utils enum() creating accepts an array
[  ERROR   ] spec/utils-enum_spec.lua @ 65: pl.utils enum() creating accepts a hash-table
[  ERROR   ] spec/utils-enum_spec.lua @ 91: pl.utils enum() creating accepts a combined array/hash-table

 7 FAILED TESTS
 3 ERRORS
alerque commented 2 years ago

I don't understand how these tests are passing on our test suite. I get different actual output for different versions of Lua: luajit gives me one this, lua5.3 something else, lua5.4 something else. The test is looking for a specific string and I don't see how we're accounting for that across our test matrix.

alerque commented 2 years ago

This seems to be a cart-before-the-horse issue that packagers just have to cope with. Our test suite is handling it by force removing the Penlight that comes with the busted installation. The Arch Linux packaging doesn't have this option and is trying to reset the Lua package path to find the currently-being-tested Penlight instead. That path wasn't being exported correctly. The test only works when the version of Penlight being used internally by busted is the same version of Penlight as is being tested.