search

nvim-lua / plenary.nvim

plenary: full; complete; entire; absolute; unqualified. All the lua functions I don't want to write twice.
MIT License
2.69k stars 284 forks source link

Are some tests expected to Fail? #576

Closed AlejandroSuero closed 5 months ago

AlejandroSuero commented 5 months ago

I was running the test and I saw some of them failing in:

Test Results ```lua ======================================== Testing: /Users/aome/dev/nvim_plugins/plenary.nvim/tests/plenary/strings_spec.lua Success || strings strdisplaywidth lua: ambiwidth = single, "abcde" -> 5 Success || strings strdisplaywidth vim: ambiwidth = single, "abcde" -> 5 Success || strings strdisplaywidth lua: ambiwidth = double, "abcde" -> 5 Success || strings strdisplaywidth vim: ambiwidth = double, "abcde" -> 5 Fail || strings strdisplaywidth lua: ambiwidth = single, "abc de" -> 10 ...nvim_plugins/plenary.nvim/tests/plenary/strings_spec.lua:20: Expected objects to be the same. Passed in: (number) 6 Expected: (number) 10 stack traceback: ...nvim_plugins/plenary.nvim/tests/plenary/strings_spec.lua:20: in function <...nvim_plugins/plenary.nvim/tests/plenary/strings_spec.lua:19> Fail || strings strdisplaywidth vim: ambiwidth = single, "abc de" -> 10 ...nvim_plugins/plenary.nvim/tests/plenary/strings_spec.lua:23: Expected objects to be the same. Passed in: (number) 6 Expected: (number) 10 stack traceback: ...nvim_plugins/plenary.nvim/tests/plenary/strings_spec.lua:23: in function <...nvim_plugins/plenary.nvim/tests/plenary/strings_spec.lua:22> Fail || strings strdisplaywidth lua: ambiwidth = double, "abc de" -> 10 ...nvim_plugins/plenary.nvim/tests/plenary/strings_spec.lua:20: Expected objects to be the same. Passed in: (number) 6 Expected: (number) 10 stack traceback: ...nvim_plugins/plenary.nvim/tests/plenary/strings_spec.lua:20: in function <...nvim_plugins/plenary.nvim/tests/plenary/strings_spec.lua:19> Fail || strings strdisplaywidth vim: ambiwidth = double, "abc de" -> 10 ...nvim_plugins/plenary.nvim/tests/plenary/strings_spec.lua:23: Expected objects to be the same. Passed in: (number) 6 Expected: (number) 10 stack traceback: ...nvim_plugins/plenary.nvim/tests/plenary/strings_spec.lua:23: in function <...nvim_plugins/plenary.nvim/tests/plenary/strings_spec.lua:22> Success || strings strdisplaywidth lua: ambiwidth = single, "アイウエオ" -> 10 Success || strings strdisplaywidth vim: ambiwidth = single, "アイウエオ" -> 10 Success || strings strdisplaywidth lua: ambiwidth = double, "アイウエオ" -> 10 Success || strings strdisplaywidth vim: ambiwidth = double, "アイウエオ" -> 10 Success || strings strdisplaywidth lua: ambiwidth = single, "├─┤" -> 3 Success || strings strdisplaywidth vim: ambiwidth = single, "├─┤" -> 3 Success || strings strdisplaywidth lua: ambiwidth = double, "├─┤" -> 6 Success || strings strdisplaywidth vim: ambiwidth = double, "├─┤" -> 6 Success || strings strdisplaywidth lua: ambiwidth = single, 123 -> 3 Success || strings strdisplaywidth vim: ambiwidth = single, 123 -> 3 Success || strings strdisplaywidth lua: ambiwidth = double, 123 -> 3 Success || strings strdisplaywidth vim: ambiwidth = double, 123 -> 3 Success || strings strcharpart lua: ("abcde", 2, nil) -> "cde" Success || strings strcharpart vim: ("abcde", 2, nil) -> "cde" Success || strings strcharpart lua: ("abcde", 2, 2) -> "cd" Success || strings strcharpart vim: ("abcde", 2, 2) -> "cd" Success || strings strcharpart lua: ("アイウエオ", 2, 2) -> "ウエ" Success || strings strcharpart vim: ("アイウエオ", 2, 2) -> "ウエ" Success || strings strcharpart lua: ("├───┤", 2, 2) -> "──" Success || strings strcharpart vim: ("├───┤", 2, 2) -> "──" Success || strings truncate ambiwidth = single, direction = right, [abcde, 6] -> abcde Success || strings truncate ambiwidth = double, direction = right, [abcde, 6] -> abcde Success || strings truncate ambiwidth = single, direction = right, [abcde, 5] -> abcde Success || strings truncate ambiwidth = double, direction = right, [abcde, 5] -> abcde Success || strings truncate ambiwidth = single, direction = right, [abcde, 4] -> abc… Success || strings truncate ambiwidth = double, direction = right, [abcde, 4] -> ab… Success || strings truncate ambiwidth = single, direction = right, [アイウエオ, 11] -> アイウエオ Success || strings truncate ambiwidth = double, direction = right, [アイウエオ, 11] -> アイウエオ Success || strings truncate ambiwidth = single, direction = right, [アイウエオ, 10] -> アイウエオ Success || strings truncate ambiwidth = double, direction = right, [アイウエオ, 10] -> アイウエオ Success || strings truncate ambiwidth = single, direction = right, [アイウエオ, 9] -> アイウエ… Success || strings truncate ambiwidth = double, direction = right, [アイウエオ, 9] -> アイウ… Success || strings truncate ambiwidth = single, direction = right, [アイウエオ, 8] -> アイウ… Success || strings truncate ambiwidth = double, direction = right, [アイウエオ, 8] -> アイウ… Success || strings truncate ambiwidth = single, direction = right, [├─┤, 7] -> ├─┤ Success || strings truncate ambiwidth = double, direction = right, [├─┤, 7] -> ├─┤ Success || strings truncate ambiwidth = single, direction = right, [├─┤, 6] -> ├─┤ Success || strings truncate ambiwidth = double, direction = right, [├─┤, 6] -> ├─┤ Success || strings truncate ambiwidth = single, direction = right, [├─┤, 5] -> ├─┤ Success || strings truncate ambiwidth = double, direction = right, [├─┤, 5] -> ├… Success || strings truncate ambiwidth = single, direction = right, [├─┤, 4] -> ├─┤ Success || strings truncate ambiwidth = double, direction = right, [├─┤, 4] -> ├… Success || strings truncate ambiwidth = single, direction = right, [├─┤, 3] -> ├─┤ Success || strings truncate ambiwidth = double, direction = right, [├─┤, 3] -> … Success || strings truncate ambiwidth = single, direction = right, [├─┤, 2] -> ├… Success || strings truncate ambiwidth = double, direction = right, [├─┤, 2] -> … Success || strings truncate ambiwidth = single, direction = left, [abcde, 6] -> abcde Success || strings truncate ambiwidth = double, direction = left, [abcde, 6] -> abcde Success || strings truncate ambiwidth = single, direction = left, [abcde, 5] -> abcde Success || strings truncate ambiwidth = double, direction = left, [abcde, 5] -> abcde Success || strings truncate ambiwidth = single, direction = left, [abcde, 4] -> …cde Success || strings truncate ambiwidth = double, direction = left, [abcde, 4] -> …de Success || strings truncate ambiwidth = single, direction = left, [アイウエオ, 11] -> アイウエオ Success || strings truncate ambiwidth = double, direction = left, [アイウエオ, 11] -> アイウエオ Success || strings truncate ambiwidth = single, direction = left, [アイウエオ, 10] -> アイウエオ Success || strings truncate ambiwidth = double, direction = left, [アイウエオ, 10] -> アイウエオ Success || strings truncate ambiwidth = single, direction = left, [アイウエオ, 9] -> …イウエオ Success || strings truncate ambiwidth = double, direction = left, [アイウエオ, 9] -> …ウエオ Success || strings truncate ambiwidth = single, direction = left, [アイウエオ, 8] -> …ウエオ Success || strings truncate ambiwidth = double, direction = left, [アイウエオ, 8] -> …ウエオ Success || strings truncate ambiwidth = single, direction = left, [├─┤, 7] -> ├─┤ Success || strings truncate ambiwidth = double, direction = left, [├─┤, 7] -> ├─┤ Success || strings truncate ambiwidth = single, direction = left, [├─┤, 6] -> ├─┤ Success || strings truncate ambiwidth = double, direction = left, [├─┤, 6] -> ├─┤ Success || strings truncate ambiwidth = single, direction = left, [├─┤, 5] -> ├─┤ Success || strings truncate ambiwidth = double, direction = left, [├─┤, 5] -> …┤ Success || strings truncate ambiwidth = single, direction = left, [├─┤, 4] -> ├─┤ Success || strings truncate ambiwidth = double, direction = left, [├─┤, 4] -> …┤ Success || strings truncate ambiwidth = single, direction = left, [├─┤, 3] -> ├─┤ Success || strings truncate ambiwidth = double, direction = left, [├─┤, 3] -> … Success || strings truncate ambiwidth = single, direction = left, [├─┤, 2] -> …┤ Success || strings truncate ambiwidth = double, direction = left, [├─┤, 2] -> … Success || strings truncate ambiwidth = single, direction = middle, [abcde, 6] -> abcde Success || strings truncate ambiwidth = double, direction = middle, [abcde, 6] -> abcde Success || strings truncate ambiwidth = single, direction = middle, [abcde, 5] -> abcde Success || strings truncate ambiwidth = double, direction = middle, [abcde, 5] -> abcde Success || strings truncate ambiwidth = single, direction = middle, [abcde, 4] -> a…de Success || strings truncate ambiwidth = double, direction = middle, [abcde, 4] -> a…e Success || strings truncate ambiwidth = single, direction = middle, [アイウエオ, 11] -> アイウエオ Success || strings truncate ambiwidth = double, direction = middle, [アイウエオ, 11] -> アイウエオ Success || strings truncate ambiwidth = single, direction = middle, [アイウエオ, 10] -> アイウエオ Success || strings truncate ambiwidth = double, direction = middle, [アイウエオ, 10] -> アイウエオ Success || strings truncate ambiwidth = single, direction = middle, [アイウエオ, 9] -> アイ…エオ Success || strings truncate ambiwidth = double, direction = middle, [アイウエオ, 9] -> ア…エオ Success || strings truncate ambiwidth = single, direction = middle, [アイウエオ, 8] -> ア…エオ Success || strings truncate ambiwidth = double, direction = middle, [アイウエオ, 8] -> ア…エオ Success || strings truncate ambiwidth = single, direction = middle, [├─┤, 7] -> ├─┤ Success || strings truncate ambiwidth = double, direction = middle, [├─┤, 7] -> ├─┤ Success || strings truncate ambiwidth = single, direction = middle, [├─┤, 6] -> ├─┤ Success || strings truncate ambiwidth = double, direction = middle, [├─┤, 6] -> ├─┤ Success || strings truncate ambiwidth = single, direction = middle, [├─┤, 5] -> ├─┤ Success || strings truncate ambiwidth = double, direction = middle, [├─┤, 5] -> …┤ Success || strings truncate ambiwidth = single, direction = middle, [├─┤, 4] -> ├─┤ Success || strings truncate ambiwidth = double, direction = middle, [├─┤, 4] -> …┤ Success || strings truncate ambiwidth = single, direction = middle, [├─┤, 3] -> ├─┤ Success || strings truncate ambiwidth = double, direction = middle, [├─┤, 3] -> … Success || strings truncate ambiwidth = single, direction = middle, [├─┤, 2] -> …┤ Success || strings truncate ambiwidth = double, direction = middle, [├─┤, 2] -> … Success || strings align_str ambiwidth = single, [abcde, 8, nil] -> "abcde " Success || strings align_str ambiwidth = double, [abcde, 8, nil] -> "abcde " Success || strings align_str ambiwidth = single, [アイウ, 8, nil] -> "アイウ " Success || strings align_str ambiwidth = double, [アイウ, 8, nil] -> "アイウ " Success || strings align_str ambiwidth = single, [├─┤, 8, nil] -> "├─┤ " Success || strings align_str ambiwidth = double, [├─┤, 8, nil] -> "├─┤ " Success || strings align_str ambiwidth = single, [abcde, 8, true] -> " abcde" Success || strings align_str ambiwidth = double, [abcde, 8, true] -> " abcde" Success || strings align_str ambiwidth = single, [アイウ, 8, true] -> " アイウ" Success || strings align_str ambiwidth = double, [アイウ, 8, true] -> " アイウ" Success || strings align_str ambiwidth = single, [├─┤, 8, true] -> " ├─┤" Success || strings align_str ambiwidth = double, [├─┤, 8, true] -> " ├─┤" Success || strings dedent tabstop = 8, empty string Success || strings dedent tabstop = 8, in case tabs are longer than spaces Success || strings dedent tabstop = 2, in case tabs are shorter than spaces Success || strings dedent tabstop = 2, ignores empty lines Success || strings dedent tabstop = 2, no indent Success || strings dedent tabstop = 2, leave_indent = 4 Success || strings dedent tabstop = 4, typical usecase: to 5 spaces Success: 121 Failed : 4 Errors : 0 ======================================== Tests Failed. Exit: 1 ```

[!important] This test doesn't when using plenary.nvim/tests/minimal_init.vim

The test for this is failing: https://github.com/nvim-lua/plenary.nvim/blob/08e301982b9a057110ede7a735dd1b5285eb341f/tests/plenary/strings_spec.lua#L5-L28

For some reason is expecting 10 characters when it only has 6 characters. If it is expecting to fail, shouldn't be better to assert that the number is not equal to the length?

Test Results ```lua ======================================== Testing: /Users/aome/dev/nvim_plugins/plenary.nvim/tests/plenary/async_testing_spec.lua Success || Async test can resume testing with vim.defer_fn Fail || Async test can resume testing from job callback ...lugins/plenary.nvim/tests/plenary/async_testing_spec.lua:13: only took 0 to get from job started to hello - expected at least 0.1 stack traceback: ...lugins/plenary.nvim/tests/plenary/async_testing_spec.lua:13: in function 'check' ...lugins/plenary.nvim/tests/plenary/async_testing_spec.lua:71: in function <...lugins/plenary.nvim/tests/plenary/async_testing_spec.lua:35> Success: 1 Failed : 1 Errors : 0 ======================================== ```

[!important] This test does fail in a different way when using plenary.nvim/tests/minimal_init.vim

Test Results minimal ```lua ======================================== Testing: /Users/aome/dev/nvim_plugins/plenary.nvim/tests/plenary/async_testing_spec.lua Fail || Async test can resume testing with vim.defer_fn ...lugins/plenary.nvim/tests/plenary/async_testing_spec.lua:13: only took 0 to get from before to after - expected at least 0.1 stack traceback: ...lugins/plenary.nvim/tests/plenary/async_testing_spec.lua:13: in function 'check' ...lugins/plenary.nvim/tests/plenary/async_testing_spec.lua:32: in function <...lugins/plenary.nvim/tests/plenary/async_testing_spec.lua:20> Fail || Async test can resume testing from job callback ...lugins/plenary.nvim/tests/plenary/async_testing_spec.lua:13: only took 0 to get from job started to hello - expected at least 0.1 stack traceback: ...lugins/plenary.nvim/tests/plenary/async_testing_spec.lua:13: in function 'check' ...lugins/plenary.nvim/tests/plenary/async_testing_spec.lua:71: in function <...lugins/plenary.nvim/tests/plenary/async_testing_spec.lua:35> Success: 0 Failed : 2 Errors : 0 ======================================== ```

The test for this is failing: https://github.com/nvim-lua/plenary.nvim/blob/08e301982b9a057110ede7a735dd1b5285eb341f/tests/plenary/async_testing_spec.lua#L19-L74

Test Results ```lua ======================================== Testing: /Users/aome/dev/nvim_plugins/plenary.nvim/tests/plenary/job_spec.lua Success || Job > cat manually > should split simple stdin Success || Job > cat manually > should allow empty strings Success || Job > cat manually > should split stdin across newlines Pending || Job > cat manually > should split stdin across newlines with no ending newline Success || Job > cat manually > should return last line when there is ending newline Pending || Job > cat manually > should return last line when there is no ending newline Success || Job env should be possible to set one env variable with an array Success || Job env should be possible to set multiple env variables with an array Success || Job env should be possible to set one env variable with a map Success || Job env should be possible to set one env variable with spaces Success || Job env should be possible to set one env variable with spaces and a map Success || Job env should be possible to set multiple env variables with a map Success || Job env should be possible to set multiple env variables with both, array and map Fail || Job > simple ls > should match systemlist ...dev/nvim_plugins/plenary.nvim/tests/plenary/job_spec.lua:303: Expected objects to be the same. Passed in: (table: 0x0105083b00) { [1] = 'total 80' [2] = '-rw-r--r--@ 1 aome staff 1067 Apr 15 15:43 LICENSE' [3] = '-rw-r--r--@ 1 aome staff 467 Apr 25 02:31 Makefile' [4] = '-rw-r--r--@ 1 aome staff 2122 Apr 15 15:43 POPUP.md' [5] = '-rw-r--r--@ 1 aome staff 11328 Apr 26 16:23 README.md' [6] = '-rw-r--r--@ 1 aome staff 4462 Apr 15 15:43 TESTS_README.md' [7] = 'drwxr-xr-x@ 3 aome staff 96 Apr 15 15:43 data' [8] = 'drwxr-xr-x@ 3 aome staff 96 Apr 15 15:43 doc' [9] = 'drwxr-xr-x@ 6 aome staff 192 Apr 26 16:23 lua' [10] = '-rw-r--r--@ 1 aome staff 704 Apr 15 15:43 plenary.nvim-scm-1.rockspec' [11] = 'drwxr-xr-x@ 3 aome staff 96 Apr 15 15:43 plugin' [12] = '-rw-r--r--@ 1 aome staff 618 Apr 15 15:43 rockspec.template' [13] = 'drwxr-xr-x@ 7 aome staff 224 Apr 25 02:31 scripts' [14] = 'drwx------@ 6 aome staff 192 Apr 26 16:23 src' [15] = 'drwxr-xr-x@ 5 aome staff 160 Apr 15 15:43 tests' *[16] = 'drwx------@ 5 aome staff 160 Apr 26 16:23 trg' } Expected: (table: 0x01050541b8) { [1] = 'total 80' [2] = '-rw-r--r--@ 1 aome staff 1067 Apr 15 15:43 LICENSE' [3] = '-rw-r--r--@ 1 aome staff 467 Apr 25 02:31 Makefile' [4] = '-rw-r--r--@ 1 aome staff 2122 Apr 15 15:43 POPUP.md' [5] = '-rw-r--r--@ 1 aome staff 11328 Apr 26 16:23 README.md' [6] = '-rw-r--r--@ 1 aome staff 4462 Apr 15 15:43 TESTS_README.md' [7] = 'drwxr-xr-x@ 3 aome staff 96 Apr 15 15:43 data' [8] = 'drwxr-xr-x@ 3 aome staff 96 Apr 15 15:43 doc' [9] = 'drwxr-xr-x@ 6 aome staff 192 Apr 26 16:23 lua' [10] = '-rw-r--r--@ 1 aome staff 704 Apr 15 15:43 plenary.nvim-scm-1.rockspec' [11] = 'drwxr-xr-x@ 3 aome staff 96 Apr 15 15:43 plugin' [12] = '-rw-r--r--@ 1 aome staff 618 Apr 15 15:43 rockspec.template' [13] = 'drwxr-xr-x@ 7 aome staff 224 Apr 25 02:31 scripts' [14] = 'drwx------@ 6 aome staff 192 Apr 26 16:23 src' [15] = 'drwxr-xr-x@ 5 aome staff 160 Apr 15 15:43 tests' *[16] = 'drwx------ 3 aome staff 96 Apr 26 16:23 trg' } stack traceback: ...dev/nvim_plugins/plenary.nvim/tests/plenary/job_spec.lua:303: in function <...dev/nvim_plugins/plenary.nvim/tests/plenary/job_spec.lua:294> Success || Job > simple ls > should match larger systemlist Success || Job > simple ls > should not timeout when completing fast jobs Success || Job > simple ls > should return the return code as well Success || Job chain should always run the next job when using and_then Success || Job chain should only run the next job on success when using and_then_on_success Success || Job chain should only run the next job on failure when using and_then_on_failure Success || Job chain should run all normal functions when using after Success || Job chain should run only on success normal functions when using after_success Success || Job chain should run only on failure normal functions when using after_failure Pending || Job .writer should allow using things like fzf Success || Job .writer should work with a table Success || Job .writer should work with a string Success || Job .writer should work with a pipe Success || Job .writer should work with a pipe, but no final newline Success || Job :wait() should respect timeout Success || Job enable_.* should not add things to results when disabled Success || Job enable_.* should not call callbacks when disabled Success || Job enable_.* should not add things to results when disabled Success || Job enable_.* should not call callbacks when disbaled Success || Job validation requires options Success || Job validation requires command Success || Job validation will not spawn jobs with invalid commands Success || Job on_exit should only be called once for wait Success || Job on_exit should only be called once for shutdown Success: 34 Failed : 1 Errors : 0 ======================================== ```

[!important] This test doesn't when using plenary.nvim/tests/minimal_init.vim

The test for this is failing: https://github.com/nvim-lua/plenary.nvim/blob/08e301982b9a057110ede7a735dd1b5285eb341f/tests/plenary/job_spec.lua#L294-L303

I don't know if some of them are expected to fail or not, and why does it differ when using the minimal setup.

If someone knows, I would like to learn why 😁

Conni2461 commented 5 months ago

all tests pass as expected. https://github.com/nvim-lua/plenary.nvim/actions/runs/8840194896/job/24275047279 also locally for me. minimalrc is required to make sure the tests work against a clean environment