SimonKagstrom / kcov

Code coverage tool for compiled programs, Python and Bash which uses debugging information to collect and report data without special compilation options
http://simonkagstrom.github.io/kcov/
GNU General Public License v2.0
721 stars 110 forks source link

Shell scripts that are called from another script between pushd and popd are not shown in coverage report #283

Open dyulu opened 5 years ago

dyulu commented 5 years ago

Shell scripts that are called from another script between pushd and popd are not shown in coverage report. To illustrate the issue, I have the following scripts in directory /root/pushd_popd/.

$ cat script01 _#!/bin/bash

pushd "/root" echo "This is script01. Calling script02..." /root/pushdpopd/script02 echo "Call script02 complete" popd

$ cat script02 _#!/bin/bash

echo "This is script02"_

Running the script: $ kcov results script01 ~ ~/pushd_popd This is script01. Calling script02... This is script02 Call script02 complete ~/pushd_popd

The coverage report showed script01 as 100%, but there was no script02 in the report.

Notes: 1) script 02 will appear in coverage report if called with dot command. 2) script02 will appear in coverage report if kcov is invoked with --bash-handle-sh-invocation even though there would be an error loading libbash_execve_redirector.so. $ kcov --bash-handle-sh-invocation results script01 ~ ~/pushd_popd ERROR: ld.so: object 'results/libbash_execve_redirector.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored. This is script01. Calling script02... This is script02 Call script02 complete ~/pushd_popd 3) Item 2 above can be a workaround, but problems come if script02 also calls another script even without pushd/popd there, say script03. In that case, script03 will not appear in coverage report.

SimonKagstrom commented 5 years ago

Hmm... can't reproduce this. For me, it works both with and without --bash-handle-sh-invocation. Are you running as root?

dyulu commented 5 years ago

Thank you very much for quick response, Simon!

Yes, I was running as root. The bash I'm using is v5.0.0(2) with the workaround mentioned in https://github.com/SimonKagstrom/kcov/issues/234 so I can get coverage data. The kcov I'm using is v34, which I installed from Gentoo ebuild. I'm running on our custom made Gentoo Linux. I'll try installing the latest kcov by cloning/building the project.

$ bash -version GNU bash, version 5.0.0(2)-release (x86_64-pc-linux-gnu) Copyright (C) 2019 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html

This is free software; you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law.

$ kcov --version kcov 34

dyulu commented 5 years ago

Updated kcov to the latest. $ kcov --version kcov v36-14-ga901

Now, kcov seems to get all scripts in current directory and sub-directories into coverage report no matter they are executed or not. The script02 did appear in the coverage report, but the coverage was 0% instead of 100% that I was looking for.

image

SimonKagstrom commented 5 years ago

Does it work if you run as non-root?

Later kcovs scan the script directory for other scripts, and adds them to the report, which is what you see here. The behavior can be turned off with --bash-dont-parse-binary-dir.

dyulu commented 5 years ago

It does not work either if run as non-root.

It is a cool feature to scan script directory for other scripts to be added to the report even if they are not executed. Thanks!

SimonKagstrom commented 5 years ago

Hmm...

What version of bash are you using? Your script examples work for me, so I guess it's something in the environment that differs.

On Sun, Feb 10, 2019, 04:24 dyulu <notifications@github.com wrote:

It does not work either if run as non-root.

It is a cool feature to scan script directory for other scripts to be added to the report even if they are not executed. Thanks!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/SimonKagstrom/kcov/issues/283#issuecomment-462100153, or mute the thread https://github.com/notifications/unsubscribe-auth/AAWyZz9455Ds1fNm8F-VfKL--U_BzgXwks5vL5DlgaJpZM4aqEKH .

dyulu commented 5 years ago

version 5.0.0(2)-release (x86_64-pc-linux-gnu). See previous comments for details.

Please let me know what version you are using and I'll try it. Thanks!

SimonKagstrom commented 5 years ago

I'm on Ubuntu 16.04, so GNU bash, version 4.3.48(1)-release (x86_64-pc-linux-gnu).

Newer bashes have some security fix which causes problems when running as root (with a possible workaround with --bash-method=DEBUG), but that should not be the problem for you since it looks the same in non-root settings.

dyulu commented 5 years ago

I found a workaround for this, i.e., to specify absolute path for output directory. For example, in this case, I need to do something like the following. kcov /root/pushd_popd/results script01

Looking at code in BashEngine::start, I think it will fix this issue if the getBaseDirectory return the absolute path instead.

std::string helperPath = IOutputHandler::getInstance().getBaseDirectory() + "bash-helper.sh"; std::string helperDebugTrapPath = IOutputHandler::getInstance().getBaseDirectory() + "bash-helper-debug-trap.sh"; std::string redirectorPath = IOutputHandler::getInstance().getBaseDirectory() + "libbash_execve_redirector.so";

SimonKagstrom commented 5 years ago

Cool! Feel free to submit a pull request, although I can't quite see why it would make a difference.