llvmbot commented 13 years ago


Bugzilla Link	9232
Resolution	FIXED
Resolved on	Mar 16, 2011 16:13
Version	trunk
OS	All
Reporter	LLVM Bugzilla Contributor
CC	@tkremenek

Extended Description

Hi,

I think either r125503 or r125611 by akirtzidis broke the 'scan-build' script, as it will still pass '-analyzer-check-idempotent-operations' and '-analyzer-check-objc-self-init' to the scanner. Please remove those invocations from scan-build to make it work again. Thanks!

tkremenek commented 13 years ago

Created an attachment (id=6324) [details] updated diff against TOT

Applied in r127758.

I'm sorry, I thought that I had tried this with a clean script and it still failed. I was especially thrown off by the error, which made it seem like '-Xclang' is an unsupported option, when in reality it was '-Xclang -o' that got parsed as unknown option. Could you check that code-path once more? Is it printing the correct offending arg?

Everything is working fine for me with your patch.

I did it mostly because I was lazy and it got the job done. That should teach me ...

No worries. If you want to disable --analyzer-display-progress, the way to do this is to:

1) add an option to scan-build, which sets an environment variable read by ccc-analyzer. 2) ccc-analyzer consults the environment variable to determine if it should not add -analyzer-display-progress to the clang -cc1 line.

There are numerous examples of this kind of argument passing in scan-build and ccc-analyzer.

I'd also like to give the user a knob to turn off the (usually benign) DeadStore reports. Perhaps I'll get to it later this week.

We recently landed changes in clang to disable/enable specific checks. We know need to forward those controls to scan-build.

llvmbot commented 13 years ago

updated diff against TOT I'm sorry, I thought that I had tried this with a clean script and it still failed. I was especially thrown off by the error, which made it seem like '-Xclang' is an unsupported option, when in reality it was '-Xclang -o' that got parsed as unknown option. Could you check that code-path once more? Is it printing the correct offending arg?

I did it mostly because I was lazy and it got the job done. That should teach me ...

I'd also like to give the user a knob to turn off the (usually benign) DeadStore reports. Perhaps I'll get to it later this week.

tkremenek commented 13 years ago

Ok, I found the problem. I have a sed script, removing the -analyzer-display-progress line from ccc-analyzer, as I cannot have the chatty output when running the analysis on the FreeBSD source tree. It makes the logfiles three times larger.

It would have been great if you had mentioned the fact you were using a sed script earlier. I spent a fair amount of time trying to diagnose this, and everyone looked fine from my end. I know this wasn't intentional, but please try and include such important details upfront in the future. Any time scan-build is run in an unconventional configuration such details should be included.

So I did it the wrong way, and this only worked by accident with the earlier command line parsing.

Please consider the following patch, which pushes Args that have to be together in one call, not split over multiple calls. Thanks

I think the patch looks fine (and a nice cleanup), but instead of using a sed script, wouldn't a better way be to provide a mechanism to disable '-analyzer-display-progress' via a scan-build option? I'm curious to know why you choose to use a sed script.

tkremenek commented 13 years ago

Created an attachment (id=6317) [details] Group some push calls, that should semantically be together

Can you regenerate this patch? It doesn't apply cleanly to TOT.

llvmbot commented 13 years ago

Group some push calls, that should semantically be together

llvmbot commented 13 years ago

Ok, I found the problem. I have a sed script, removing the -analyzer-display-progress line from ccc-analyzer, as I cannot have the chatty output when running the analysis on the FreeBSD source tree. It makes the logfiles three times larger.

So I did it the wrong way, and this only worked by accident with the earlier command line parsing.

Please consider the following patch, which pushes Args that have to be together in one call, not split over multiple calls. Thanks

tkremenek commented 13 years ago

Of course I tried that, and passing the minimal amount of flags to ccc-analyzer still breaks when clang tries to parse them. That's why I specifically asked you to provide the exact arguments that it is passing to clang in your case, so I can compare the two invocations.

All I am doing is:

$ touch t.c $ scan-build -v -v -v gcc -c t.c ...

SHELL (cd '/private/tmp' && '/usr/bin/clang' '-cc1' '-triple' 'x86_64-apple-darwin10.0.0' '-analyze' '-disable-free' '-disable-llvm-verifier' '-main-file-name' 't.c' '-analyzer-store=region' '-analyzer-opt-analyze-nested-blocks' '-analyzer-check-dead-stores' '-analyzer-check-objc-mem' '-analyzer-eagerly-assume' '-analyzer-check-objc-methodsigs' '-analyzer-check-objc-unused-ivars' '-analyzer-check-idempotent-operations' '-analyzer-output' 'plist' '-w' '-pic-level' '1' '-mdisable-fp-elim' '-masm-verbose' '-munwind-tables' '-target-cpu' 'core2' '-target-linker-version' '123.2' '-resource-dir' '/usr/bin/../lib/clang/2.0' '-ferror-limit' '19' '-fmessage-length' '0' '-stack-protector' '1' '-fblocks' '-fdiagnostics-show-option' '-analyzer-display-progress' '-analyzer-output=html' '-o' '/var/folders/UP/UP-KEPSNERaelthyiIare++++TI/-Tmp-/scan-build-2011-03-10-3' '-x' 'c' 't.c')

scan-build: Removing directory '/var/folders/UP/UP-KEPSNERaelthyiIare++++TI/-Tmp-/scan-build-2011-03-10-3' because it contains no reports.

tkremenek commented 13 years ago

The -cc1 arguments are constructed in the 'Analyze' function in ccc-analyzer. That's where the -Xclang options get passed. Could you try something?

1) Update to TOT. I just committed some ccc-analyzer changes (as a sanity check).

2) Within ccc-analyzer, after the following code:

# Display Ubiviz graph?
if (defined $ENV{'CCC_UBI'}) {   
  push @&#8203;Args, "-Xclang";
  push @&#8203;Args,"-analyzer-viz-egraph-ubigraph";
}

please add:

print "original command line: @Args\n";

and then comment on what that prints out.

tkremenek commented 13 years ago

Of course I tried that, and passing the minimal amount of flags to ccc-analyzer still breaks when clang tries to parse them. That's why I specifically asked you to provide the exact arguments that it is passing to clang in your case, so I can compare the two invocations.

% echo "int main(void) { return 0; }" >! foo.c % env CCC_ANALYZER_LOG=1 CLANG=/data/src/llvm/Release/bin/clang /data/src/llvm/Release/bin/ccc-analyzer foo.c gcc foo.c

SHELL (cd '/tmp' && '/data/src/llvm/Release/bin/clang' '-cc1' '-triple'

'x86_64-unknown-freebsd8.2' '-analyze' '-disable-free' '-disable-llvm-verifier' '-main-file-name' 'foo.c' '-analyzer-store=region' '-analyzer-opt-analyze-nested-blocks' '-analyzer-checker=core' '-analyzer-checker=unix' '-analyzer-checker=DeadStores' '-analyzer-eagerly-assume' '-analyzer-output' 'plist' '-w' '-mrelocation-model' 'static' '-mdisable-fp-elim' '-masm-verbose' '-mconstructor-aliases' '-munwind-tables' '-target-cpu' 'x86-64' '-target-linker-version' '2.15' '-momit-leaf-frame-pointer' '-resource-dir' '/data/src/llvm/Release/bin/../lib/clang/3.0' '-ferror-limit' '19' '-fmessage-length' '0' '-fgnu-runtime' '-fdiagnostics-show-option' '-Xclang' '-o' 'foo.plist' '-x' 'c' 'foo.c') error: unknown argument: '-Xclang'

Now when I bastardize the clang line like this, it does something, but no plist files are generated so it is still not really useful.

'/data/src/llvm/Release/bin/clang' '-analyze' '-disable-free' '-disable-llvm-verifier' '-main-file-name' 'foo.c' '-analyzer-store=region' '-analyzer-opt-analyze-nested-blocks' '-analyzer-checker=core' '-analyzer-checker=unix' '-analyzer-checker=DeadStores' '-analyzer-eagerly-assume' '-Xclang' '-x' 'foo.c' clang: warning: argument unused during compilation: '-analyze' clang: warning: argument unused during compilation: '-disable-free' clang: warning: argument unused during compilation: '-disable-llvm-verifier' clang: warning: argument unused during compilation: '-analyzer-store=region' clang: warning: argument unused during compilation: '-analyzer-opt-analyze-nested-blocks' clang: warning: argument unused during compilation: '-analyzer-checker=core' clang: warning: argument unused during compilation: '-analyzer-checker=unix' clang: warning: argument unused during compilation: '-analyzer-checker=DeadStores' clang: warning: argument unused during compilation: '-analyzer-eagerly-assume' error: error reading '/tmp/cc-y099HN.o' 1 error generated.

So, what is -cc1 doing and what is -Xclang supposed to achieve, exactly?

-cc1 are the low level driver arguments. Invoking 'clang' without -cc1 causes the driver to lower the "high-level" arguments to the low-level (-cc1) arguments, and then that's how the actual compiler get's run. With the high-level arguments, the driver determines things like target triple automatically, and then explicitly forwards those decisions to -cc1 arguments. That gives us a portable way to run the compiler with exactly the same behavior on different machines.

-Xclang is way to explicitly pass -cc1 arguments using the high-level driver mechanism. E.g.

$ clang -fsyntax-only -Xclang -analyze t.c -###

"/usr/bin/clang" "-cc1" "-triple" "x86_64-apple-darwin10.0.0" "-fsyntax-only" "-disable-free" "-disable-llvm-verifier" "-main-file-name" "t.c" "-pic-level" "1" "-mdisable-fp-elim" "-masm-verbose" "-munwind-tables" "-target-cpu" "core2" "-target-linker-version" "123.2" "-resource-dir" "/usr/bin/../lib/clang/2.0" "-ferror-limit" "19" "-fmessage-length" "270" "-stack-protector" "1" "-fblocks" "-fdiagnostics-show-option" "-fcolor-diagnostics" "-analyze" "-x" "c" "t.c"

Notice the '-analyze' at the end. The -Xclang arguments are munged onto the the regular arguments by ccc-analyzer to pass options to the static analyzer. -Xclang doesn't make sense for the low-level driver, just the high-level driver.

So I suspect that some argument munging within ccc-analyzer isn't working correctly, although it works fine on my system.

llvmbot commented 13 years ago

Of course I tried that, and passing the minimal amount of flags to ccc-analyzer still breaks when clang tries to parse them. That's why I specifically asked you to provide the exact arguments that it is passing to clang in your case, so I can compare the two invocations.

% echo "int main(void) { return 0; }" >! foo.c % env CCC_ANALYZER_LOG=1 CLANG=/data/src/llvm/Release/bin/clang /data/src/llvm/Release/bin/ccc-analyzer foo.c gcc foo.c

SHELL (cd '/tmp' && '/data/src/llvm/Release/bin/clang' '-cc1' '-triple' 'x86_64-unknown-freebsd8.2' '-analyze' '-disable-free' '-disable-llvm-verifier' '-main-file-name' 'foo.c' '-analyzer-store=region' '-analyzer-opt-analyze-nested-blocks' '-analyzer-checker=core' '-analyzer-checker=unix' '-analyzer-checker=DeadStores' '-analyzer-eagerly-assume' '-analyzer-output' 'plist' '-w' '-mrelocation-model' 'static' '-mdisable-fp-elim' '-masm-verbose' '-mconstructor-aliases' '-munwind-tables' '-target-cpu' 'x86-64' '-target-linker-version' '2.15' '-momit-leaf-frame-pointer' '-resource-dir' '/data/src/llvm/Release/bin/../lib/clang/3.0' '-ferror-limit' '19' '-fmessage-length' '0' '-fgnu-runtime' '-fdiagnostics-show-option' '-Xclang' '-o' 'foo.plist' '-x' 'c' 'foo.c')

error: unknown argument: '-Xclang'

Now when I bastardize the clang line like this, it does something, but no plist files are generated so it is still not really useful.

'/data/src/llvm/Release/bin/clang' '-analyze' '-disable-free' '-disable-llvm-verifier' '-main-file-name' 'foo.c' '-analyzer-store=region' '-analyzer-opt-analyze-nested-blocks' '-analyzer-checker=core' '-analyzer-checker=unix' '-analyzer-checker=DeadStores' '-analyzer-eagerly-assume' '-Xclang' '-x' 'foo.c' clang: warning: argument unused during compilation: '-analyze' clang: warning: argument unused during compilation: '-disable-free' clang: warning: argument unused during compilation: '-disable-llvm-verifier' clang: warning: argument unused during compilation: '-analyzer-store=region' clang: warning: argument unused during compilation: '-analyzer-opt-analyze-nested-blocks' clang: warning: argument unused during compilation: '-analyzer-checker=core' clang: warning: argument unused during compilation: '-analyzer-checker=unix' clang: warning: argument unused during compilation: '-analyzer-checker=DeadStores' clang: warning: argument unused during compilation: '-analyzer-eagerly-assume' error: error reading '/tmp/cc-y099HN.o' 1 error generated.

So, what is -cc1 doing and what is -Xclang supposed to achieve, exactly?

tkremenek commented 13 years ago

I tried on Ubuntu as well, with the same end result. Can you show me the arguments that are passed to clang for your setup? It would be nice to have a working counter-example ...

What specifically did you try? My question was if you remove some of the compiler flags, does the issue persist? On my setup, the -march flag is not passed, so I'm wondering if that is (possibly) the issue.

Specifically, if you take the following line:

env CCC_ANALYZER_LOG=1 CLANG=/data/src/llvm/Release/bin/clang /data/src/llvm/Release/bin/ccc-analyzer -O2 -pipe -march=native -g -W -Wall -Wstrict-prototypes -Wno-unused-parameter -Wwrite-strings -DVERSION="\"1.10.9\"" -DHAVE_CONFIG_H -c main.c gcc -O2 -pipe -march=native -g -W -Wall -Wstrict-prototypes -Wno-unused-parameter -Wwrite-strings -DVERSION="1.10.9" -DHAVE_CONFIG_H -c main.c

and remove various flags, e.g., -march=native, does ccc-analyzer analyze your code?

llvmbot commented 13 years ago

I tried on Ubuntu as well, with the same end result. Can you show me the arguments that are passed to clang for your setup? It would be nice to have a working counter-example ...

tkremenek commented 13 years ago

I'm using TOT llvm, clang, ccc-analyzer and scan-build.

% Release/bin/clang --version clang version 3.0 (trunk 127234) Target: x86_64-unknown-freebsd8.2 Thread model: posix % svn info ./tools/clang/tools/scan-build/ccc-analyzer Path: tools/clang/tools/scan-build/ccc-analyzer Name: ccc-analyzer URL: http://llvm.org/svn/llvm-project/cfe/trunk/tools/scan-build/ccc-analyzer Repository Root: http://llvm.org/svn/llvm-project Repository UUID: 91177308-0d34-0410-b5e6-96231b3b80d8 Revision: 127234 Node Kind: file Schedule: normal Last Changed Author: kremenek Last Changed Rev: 125712 Last Changed Date: 2011-02-17 03:28:30 +0100 (Thu, 17 Feb 2011) Text Last Updated: 2011-03-08 10:20:35 +0100 (Tue, 08 Mar 2011) Checksum: db09cfe135ee490b7f75ed41dd194c54

Here's a direct invocation:

% env CCC_ANALYZER_LOG=1 CLANG=/data/src/llvm/Release/bin/clang /data/src/llvm/Release/bin/ccc-analyzer -O2 -pipe -march=native -g -W -Wall -Wstrict-prototypes -Wno-unused-parameter -Wwrite-strings -DVERSION="\"1.10.9\"" -DHAVE_CONFIG_H -c main.c gcc -O2 -pipe -march=native -g -W -Wall -Wstrict-prototypes -Wno-unused-parameter -Wwrite-strings -DVERSION="1.10.9" -DHAVE_CONFIG_H -c main.c

Using this example, can you try winnowing down the arguments? For example, remove -march=native and see if it works. Not all of these arguments are valid on my setup, so I cannot 100% reproduce what you are seeing. If the problem persists, even with all the arguments minimized, the changes the line of investigation.

llvmbot commented 13 years ago

I'm using TOT llvm, clang, ccc-analyzer and scan-build.

% Release/bin/clang --version clang version 3.0 (trunk 127234) Target: x86_64-unknown-freebsd8.2 Thread model: posix % svn info ./tools/clang/tools/scan-build/ccc-analyzer Path: tools/clang/tools/scan-build/ccc-analyzer Name: ccc-analyzer URL: http://llvm.org/svn/llvm-project/cfe/trunk/tools/scan-build/ccc-analyzer Repository Root: http://llvm.org/svn/llvm-project Repository UUID: 91177308-0d34-0410-b5e6-96231b3b80d8 Revision: 127234 Node Kind: file Schedule: normal Last Changed Author: kremenek Last Changed Rev: 125712 Last Changed Date: 2011-02-17 03:28:30 +0100 (Thu, 17 Feb 2011) Text Last Updated: 2011-03-08 10:20:35 +0100 (Tue, 08 Mar 2011) Checksum: db09cfe135ee490b7f75ed41dd194c54

Here's a direct invocation:

% env CCC_ANALYZER_LOG=1 CLANG=/data/src/llvm/Release/bin/clang /data/src/llvm/Release/bin/ccc-analyzer -O2 -pipe -march=native -g -W -Wall -Wstrict-prototypes -Wno-unused-parameter -Wwrite-strings -DVERSION="\"1.10.9\"" -DHAVE_CONFIG_H -c main.c gcc -O2 -pipe -march=native -g -W -Wall -Wstrict-prototypes -Wno-unused-parameter -Wwrite-strings -DVERSION="1.10.9" -DHAVE_CONFIG_H -c main.c

SHELL (cd '/data/src/mdocml' && '/data/src/llvm/Release/bin/clang' '-cc1' '-triple' 'x86_64-unknown-freebsd8.2' '-analyze' '-disable-free' '-disable-llvm-verifier' '-main-file-name' 'main.c' '-analyzer-store=region' '-analyzer-opt-analyze-nested-blocks' '-analyzer-checker=core' '-analyzer-checker=unix' '-analyzer-checker=DeadStores' '-analyzer-eagerly-assume' '-analyzer-output' 'plist' '-w' '-mrelocation-model' 'static' '-mdisable-fp-elim' '-masm-verbose' '-mconstructor-aliases' '-munwind-tables' '-target-cpu' 'x86-64' '-target-linker-version' '2.15' '-momit-leaf-frame-pointer' '-resource-dir' '/data/src/llvm/Release/bin/../lib/clang/3.0' '-D' 'VERSION="1.10.9"' '-D' 'HAVE_CONFIG_H' '-Wno-unused-parameter' '-ferror-limit' '19' '-fmessage-length' '0' '-fgnu-runtime' '-fdiagnostics-show-option' '-Xclang' '-o' 'main.plist' '-x' 'c' 'main.c')

error: unknown argument: '-Xclang'

The clang call fails, I'm not sure which one of the options is to blame, but some more minimal examples seem to do something (not something very useful, though)

% '/data/src/llvm/Release/bin/clang' '-Xclang' '-x' 'c' 'main.c' clang: error: no such file or directory: 'c' (1)% '/data/src/llvm/Release/bin/clang' '-Xclang' 'main.c' clang: error: no input files (1)% '/data/src/llvm/Release/bin/clang' '-Xclang' 'main.plist' '-x' 'c' 'main.c' error: error reading 'main.plist' main.c:326:36: error: use of undeclared identifier 'VERSION' (void)printf("%s %s\n", progname, VERSION); ^ 2 errors generated. (1)% '/data/src/llvm/Release/bin/clang' '-D' 'VERSION="1.10.9"' '-D' 'HAVE_CONFIG_H' '-Xclang' 'main.plist' '-x' 'c' 'main.c' error: error reading 'main.plist' 1 error generated. (1)% '/data/src/llvm/Release/bin/clang' '-D' 'VERSION="1.10.9"' '-D' 'HAVE_CONFIG_H' '-Xclang' '-o' 'main.plist' '-x' 'c' 'main.c' clang: error: no such file or directory: 'main.plist' (1)% touch main.plist % '/data/src/llvm/Release/bin/clang' '-D' 'VERSION="1.10.9"' '-D' 'HAVE_CONFIG_H' '-Xclang' '-o' 'main.plist' '-x' 'c' 'main.c' error: error reading '/tmp/cc-cIYSpI.o' 1 error generated.

tkremenek commented 13 years ago

scan-build is still broken for me like this:

error: unknown argument: '-Xclang'

so the analyzer won't get run at all when scan-build is used.

Are you using scan-build with an older version of Clang? It will only work with mainline SVN right now.

llvmbot commented 13 years ago

scan-build is still broken for me like this:

error: unknown argument: '-Xclang'

so the analyzer won't get run at all when scan-build is used.

tkremenek commented 13 years ago

Proposed patch (sorry for c&p, cannot create the file properly right now ...)

Index: scan-build

--- scan-build (revision 125662) +++ scan-build (working copy) @@ -117,12 +117,10 @@ '-analyzer-check-dead-stores' => 1, '-analyzer-check-objc-mem' => 1, '-analyzer-check-objc-methodsigs' => 1,

'-analyzer-check-objc-self-init' => 1,
Do not enable the missing -dealloc check by default.

'-analyzer-check-objc-missing-dealloc' => 1,

'-analyzer-check-objc-unused-ivars' => 1, '-analyzer-check-security-syntactic' => 1,

'-analyzer-check-idempotent-operations' => 1 );

----------------------------------------------------------------------------

We're completely revamping the checker arguments right now. scan-build will remain temporarily broken (hopefully just a couple more days) until the dust settles there. Most of these arguments will go away and be replaced with new ones.

llvmbot commented 13 years ago

Proposed patch (sorry for c&p, cannot create the file properly right now ...)

Index: scan-build

--- scan-build (revision 125662) +++ scan-build (working copy) @@ -117,12 +117,10 @@ '-analyzer-check-dead-stores' => 1, '-analyzer-check-objc-mem' => 1, '-analyzer-check-objc-methodsigs' => 1,

'-analyzer-check-objc-self-init' => 1,
Do not enable the missing -dealloc check by default.

'-analyzer-check-objc-missing-dealloc' => 1,

'-analyzer-check-objc-unused-ivars' => 1, '-analyzer-check-security-syntactic' => 1,
'-analyzer-check-idempotent-operations' => 1 );

----------------------------------------------------------------------------

llvmbot commented 13 years ago

assigned to @tkremenek

llvm / llvm-project

Remove deprecated/removed scanner arguments from scan-build #9604

Extended Description

SHELL (cd '/tmp' && '/data/src/llvm/Release/bin/clang' '-cc1' '-triple'

Index: scan-build

Do not enable the missing -dealloc check by default.

'-analyzer-check-objc-missing-dealloc' => 1,

----------------------------------------------------------------------------

Index: scan-build

Do not enable the missing -dealloc check by default.

'-analyzer-check-objc-missing-dealloc' => 1,

----------------------------------------------------------------------------