search

SimonKagstrom / kcov

Code coverage tool for compiled programs, Python and Bash which uses debugging information to collect and report data without special compilation options
http://simonkagstrom.github.io/kcov/
GNU General Public License v2.0
709 stars 109 forks source link

kcov enters infinite loop when run with bats executable #458

Open twils0 opened 1 month ago

twils0 commented 1 month ago

When I run kcov (as detailed below), it enters an infinite loop. I can't seem to get kcov to honor my exclude/include flags. I've tried many different variations of {include|exclude}-{path|pattern}.

I installed executables for kcov and bats in my project. I want to get the coverage for script.sh, based on the bats test script_test.bats.

Here's an example of the commands I'm tried:

kcov/bin/src/kcov --bash-dont-parse-binary-dir --exclude-pattern=bats/ --include-pattern=script.sh coverage/ ./bats/bin/bats script_tests.bats

Here's my project structure:

project/
project/script.sh
project/script_test.bats
project/bats/bin/bat    <= bats executable
project/bats/bats-core
project/bats/bats-support
project/bats/bats-assert
project/kcov/*  <= cloned from Github
project/kcov/bin/src/kcov    <= kcov executable

Here's the last few lines of the infinite loop that kcov enters into. I can't get kcov to ignore these files related to the bats executable.

I've also downloaded bats through Homebrew. When I run kcov against the bats executable from Homebrew, this infinite loop again occurs with paths pointing at the bats Homebrew directory, instead of the bats directory in my project (as shown below).

# kcov@project/bats/lib/bats-core/common.bash@5@printf '# %s\n' 'kcov@project/bats/lib/bats-core/common.bash@5@printf '\''# %s\n'\'' '\''kkcov@project/bats/lib/bats-core/tracing.bash@281@for path in "${BATS_DEBUG_EXCLUDE_PATHS[@]}"'\'''
# kcov@project/bats/lib/bats-core/common.bash@15@printf '%s\n' '# kcov@project/bats/lib/bats-core/common.bash@15@printf '\''%s\n'\'' '\''# kkcov@project/bats/lib/bats-core/tracing.bash@281@for path in "${BATS_DEBUG_EXCLUDE_PATHS[@]}"'\'''
# kcov@project/bats/lib/bats-core/common.bash@14@read -r line
# kcov@project/bats/lib/bats-core/common.bash@15@printf '%s\n' '# kcov@project/bats/lib/bats-core/common.bash@4@IFS='
# kcov@project/bats/lib/bats-core/common.bash@14@read -r line
# kcov@project/bats/lib/bats-core/common.bash@4@IFS=
# kcov@project/bats/lib/bats-core/common.bash@4@read -r line
# kcov@project/bats/lib/bats-core/common.bash@5@printf '# %s\n' 'kcov@project/bats/lib/bats-core/common.bash@15@printf '\''%s\n'\'' '\''# kkcov@project/bats/lib/bats-core/tracing.bash@195@[[ 0 -gt 0 ]]'\'''
# kcov@project/bats/lib/bats-core/common.bash@15@printf '%s\n' '# kcov@project/bats/lib/bats-core/common.bash@14@read -r line'
# kcov@project/bats/lib/bats-core/common.bash@14@read -r line
# kcov@project/bats/lib/bats-core/common.bash@15@printf '%s\n' '# kcov@project/bats/lib/bats-core/common.bash@4@read -r line'
# kcov@project/bats/lib/bats-core/common.bash@14@read -r line
# kcov@project/bats/lib/bats-core/common.bash@4@IFS=
# kcov@project/bats/lib/bats-core/common.bash@4@read -r line
# kcov@project/bats/lib/bats-core/common.bash@5@printf '# %s\n' kcov@project/bats/lib/bats-core/common.bash@4@IFS=
# kcov@project/bats/lib/bats-core/common.bash@4@IFS=
# kcov@project/bats/lib/bats-core/common.bash@4@read -r line
# kcov@project/bats/lib/bats-core/common.bash@5@printf '# %s\n' 'kcov@project/bats/lib/bats-core/common.bash@14@read -r line'
# kcov@project/bats/lib/bats-core/common.bash@4@IFS=
# kcov@project/bats/lib/bats-core/common.bash@4@read -r line
# kcov@project/bats/lib/bats-core/common.bash@15@printf '%s\n' '# kcov@project/bats/lib/bats-core/common.bash@5@printf '\''# %s\n'\'' kcov@project/script_tests.bats@42@DEPLOY_END=2022-01-01T00:00:00Z'
# kcov@project/bats/lib/bats-core/common.bash@14@read -r line
^C# kcov@project/bats/lib/bats-core/common.bash@5@printf '# %s\n' 'kcov@project/bats/lib/bats-core/common.bash@4@read -r line'
SimonKagstrom commented 1 month ago

Strange. Does kcov work with any bash script in your installation?

One thing you can try, although I don't think it will give that much more information, is to run with --debug=31. With some luck, it might indicate where it gets stuck.

twils0 commented 1 month ago

Thank you for your fast reply!

My question overall is: why can't I exclude the bats binary (executable) directory and/or the bats directory? I'm using --bash-dont-parse-binary-dir --exclude-pattern=bats/. I've tried countless variations of the include and exclude commands.

If I run kcov against script_cli.sh directly (just a bash script not a bats test obviously), it seems to work. No infinite loop. It prints out some lines of code from the script and logs that look like the following.

0x7f9bb29041b0 REPORT hit at 0x1b4098079bf
0x7f9bb2804080 REPORT hit at 0x1b5aa9424c3
0x7f9bb29041b0 REPORT hit at 0x1b5098079bf

I ran with "--debug=31". I'll post the repeating logs from the end of kcov's output below. The logs seem different, but I'm not sure if they will help. From what I can tell, it looks like kcov prints some logs related to script_cli.sh and script_cli_test.bats at the beginning, and it then starts printing out logs related to bats - common.bash, tracing.bash, etc. - in an infinite loop.

I'm sure that it's related to the bats executable that I pass to kcov.

# kkcov@project/bats/lib/bats-core/tracing.bash@17@local limit=6
# kkcov@project/bats/lib/bats-core/tracing.bash@19@(( i = 2 ))
# kkcov@project/bats/lib/bats-core/tracing.bash@19@(( i < limit  ))
# kkcov@project/bats/lib/bats-core/tracing.bash@22@test_file=project/bats/bats-support/src/output.bash
# kkcov@project/bats/lib/bats-core/tracing.bash@23@funcname=batslib_is_single_line
# kkcov@project/bats/lib/bats-core/tracing.bash@24@BATS_DEBUG_LAST_STACK_TRACE+=("${BASH_LINENO[$((i - 1))]} $funcname $test_file")
# kkcov@project/bats/lib/bats-core/tracing.bash@19@(( ++i ))
# kkcov@project/bats/lib/bats-core/tracing.bash@19@(( i < limit  ))
# kkcov@project/bats/lib/bats-core/tracing.bash@22@test_file=project/bats/bats-support/src/output.bash
# kkcov@project/bats/lib/bats-core/tracing.bash@23@funcname=batslib_print_kv_single_or_multi
# kkcov@project/bats/lib/bats-core/tracing.bash@24@BATS_DEBUG_LAST_STACK_TRACE+=("${BASH_LINENO[$((i - 1))]} $funcname $test_file")
# kkcov@project/bats/lib/bats-core/tracing.bash@19@(( ++i ))
# kkcov@project/bats/lib/bats-core/tracing.bash@19@(( i < limit  ))
# kkcov@project/bats/lib/bats-core/tracing.bash@22@test_file=project/bats/bats-assert/src/assert.bash
# kkcov@project/bats/lib/bats-core/tracing.bash@23@funcname=assert_equal
# kkcov@project/bats/lib/bats-core/tracing.bash@24@BATS_DEBUG_LAST_STACK_TRACE+=("${BASH_LINENO[$((i - 1))]} $funcname $test_file")
# kkcov@project/bats/lib/bats-core/tracing.bash@19@(( ++i ))
# kkcov@project/bats/lib/bats-core/tracing.bash@19@(( i < limit  ))
# kkcov@project/bats/lib/bats-core/tracing.bash@22@test_file=project/script_cli_tests.bats
# kkcov@project/bats/lib/bats-core/tracing.bash@23@funcname=test_Missing_DEPLOY-2d5fMETHOD
# kkcov@project/bats/lib/bats-core/tracing.bash@24@BATS_DEBUG_LAST_STACK_TRACE+=("${BASH_LINENO[$((i - 1))]} $funcname $test_file")
# kkcov@project/bats/lib/bats-core/tracing.bash@19@(( ++i ))
# kkcov@project/bats/lib/bats-core/tracing.bash@19@(( i < limit  ))
# kkcov@project/bats/lib/bats-core/tracing.bash@298@bats_emit_trace
# kkcov@project/bats/lib/bats-core/tracing.bash@195@[[ 0 -gt 0 ]]
# kkcov@project/bats/bats-support/src/output.bash@82@bats_debug_trap project/bats/bats-support/src/output.bash
# kkcov@project/bats/lib/bats-core/tracing.bash@278@local NORMALIZED_INPUT
# kkcov@project/bats/lib/bats-core/tracing.bash@279@bats_normalize_windows_dir_path NORMALIZED_INPUT project/bats/bats-support/src
# kkcov@project/bats/lib/bats-core/tracing.bash@161@local output_var=NORMALIZED_INPUT path=project/bats/bats-support/src
# kkcov@project/bats/lib/bats-core/tracing.bash@162@[[ NORMALIZED_INPUT != NORMALIZED_INPUT ]]
# kkcov@project/bats/lib/bats-core/tracing.bash@165@[[ project/bats/bats-support/src == ?:* ]]
# kkcov@project/bats/lib/bats-core/tracing.bash@171@NORMALIZED_INPUT=project/bats/bats-support/src
# kkcov@project/bats/lib/bats-core/tracing.bash@173@printf -v NORMALIZED_INPUT %s project/bats/bats-support/src
# kkcov@project/bats/lib/bats-core/tracing.bash@280@local path
# kkcov@project/bats/lib/bats-core/tracing.bash@281@for path in "${BATS_DEBUG_EXCLUDE_PATHS[@]}"
# kkcov@project/bats/lib/bats-core/tracing.bash@282@[[ project/bats/bats-support/src == /\p\r\o\j\e\c\t/\b\a\t\s\/\l\i\b\/* ]]
# kkcov@project/bats/lib/bats-core/tracing.bash@281@for path in "${BATS_DEBUG_EXCLUDE_PATHS[@]}"
# kkcov@project/bats/lib/bats-core/tracing.bash@282@[[ project/bats/bats-support/src == /\p\r\o\j\e\c\t/\b\a\t\s\/\l\i\b\/* ]]
# kkcov@project/bats/lib/bats-core/tracing.bash@281@for path in "${BATS_DEBUG_EXCLUDE_PATHS[@]}"
# kkcov@project/bats/lib/bats-core/tracing.bash@282@[[ project/bats/bats-support/src == /\p\r\o\j\e\c\t/\b\a\t\s\/\l\i\b\/* ]]
# kkcov@project/bats/lib/bats-core/tracing.bash@290@[[ NOTSET == NOTSET ]]
# kkcov@project/bats/lib/bats-core/tracing.bash@290@[[ NOTSET == NOTSET ]]
# kkcov@project/bats/lib/bats-core/tracing.bash@293@BATS_DEBUG_LASTLAST_STACK_TRACE=(${BATS_DEBUG_LAST_STACK_TRACE[@]+"${BATS_DEBUG_LAST_STACK_TRACE[@]}"})
# kkcov@project/bats/lib/bats-core/tracing.bash@295@BATS_DEBUG_LAST_LINENO=(${BASH_LINENO[@]+"${BASH_LINENO[@]}"})
# kkcov@project/bats/lib/bats-core/tracing.bash@296@BATS_DEBUG_LAST_SOURCE=(${BASH_SOURCE[@]+"${BASH_SOURCE[@]}"})
# kkcov@project/bats/lib/bats-core/tracing.bash@297@bats_capture_stack_trace
# kkcov@project/bats/lib/bats-core/tracing.bash@12@local test_file
# kkcov@project/bats/lib/bats-core/tracing.bash@13@local funcname
# kkcov@project/bats/lib/bats-core/tracing.bash@14@local i
# kkcov@project/bats/lib/bats-core/tracing.bash@16@BATS_DEBUG_LAST_STACK_TRACE=()
# kkcov@project/bats/lib/bats-core/tracing.bash@17@local limit=6
# kkcov@project/bats/lib/bats-core/tracing.bash@19@(( i = 2 ))
# kkcov@project/bats/lib/bats-core/tracing.bash@19@(( i < limit  ))
# kkcov@project/bats/lib/bats-core/tracing.bash@22@test_file=project/bats/bats-support/src/output.bash
# kkcov@project/bats/lib/bats-core/tracing.bash@23@funcname=batslib_is_single_line
# kkcov@project/bats/lib/bats-core/tracing.bash@24@BATS_DEBUG_LAST_STACK_TRACE+=("${BASH_LINENO[$((i - 1))]} $funcname $test_file")
# kkcov@project/bats/lib/bats-core/tracing.bash@19@(( ++i ))
# kkcov@project/bats/lib/bats-core/tracing.bash@19@(( i < limit  ))
# kkcov@project/bats/lib/bats-core/tracing.bash@22@test_file=project/bats/bats-support/src/output.bash
# kkcov@project/bats/lib/bats-core/tracing.bash@23@funcname=batslib_print_kv_single_or_multi
# kkcov@project/bats/lib/bats-core/tracing.bash@24@BATS_DEBUG_LAST_STACK_TRACE+=("${BASH_LINENO[$((i - 1))]} $funcname $test_file")
# kkcov@project/bats/lib/bats-core/tracing.bash@19@(( ++i ))
# kkcov@project/bats/lib/bats-core/tracing.bash@19@(( i < limit  ))
# kkcov@project/bats/lib/bats-core/tracing.bash@22@test_file=project/bats/bats-assert/src/assert.bash
# kkcov@project/bats/lib/bats-core/tracing.bash@23@funcname=assert_equal
# kkcov@project/bats/lib/bats-core/tracing.bash@24@BATS_DEBUG_LAST_STACK_TRACE+=("${BASH_LINENO[$((i - 1))]} $funcname $test_file")
# kkcov@project/bats/lib/bats-core/tracing.bash@19@(( ++i ))
# kkcov@project/bats/lib/bats-core/tracing.bash@19@(( i < limit  ))
# kkcov@project/bats/lib/bats-core/tracing.bash@22@test_file=project/script_cli_tests.bats
# kkcov@project/bats/lib/bats-core/tracing.bash@23@funcname=test_Missing_DEPLOY-2d5fMETHOD
# kkcov@project/bats/lib/bats-core/tracing.bash@24@BATS_DEBUG_LAST_STACK_TRACE+=("${BASH_LINENO[$((i - 1))]} $funcname $test_file")
# kkcov@project/bats/lib/bats-core/tracing.bash@19@(( ++i ))
# kkcov@project/bats/lib/bats-core/tracing.bash@19@(( i < limit  ))
# kkcov@project/bats/lib/bats-core/tracing.bash@298@bats_emit_trace
^C# kkcov@project/bats/lib/bats-core/tracing.bash@195@[[ 0 -gt 0 ]]
twils0 commented 1 month ago

I can tell one thing from the logs above. This line references the first test in my bats test file:

# kkcov@project/bats/lib/bats-core/tracing.bash@23@funcname=test_Missing_DEPLOY-2d5fMETHOD

My first test is called: Missing DEPLOY_METHOD. It prints out error messages, if my script is missing the DEPLOY_METHOD environment variable. The test provides all required environment variables, except DEPLOY_METHOD, and looks for the correct error messages from the script output.