Closed bouillon closed 5 months ago
Hi @bouillon 👋
The issue you linked to discusses how to filter out queries when running CodeQL in a GitHub Actions workflow. For usage with the CLI, you can put together a custom query suite. There is an example of what such a file would look like to filter out some queries from an existing suite.
Thanks. I created a file codeqlfilter.qls with very simple content
- qlpack: codeql/python-queries
I would expect this a call like
codeql database analyze /tmp/codeql_db/ codeqlfilter.qls --format csv --output /tmp/codeql-report.csv --download --threads=12
produce me the same result as without codeqlfilter.qls but a get much less results (only errors)
all recommendations, like missing doc-string are missing.
$ codeql resolve queries codeqlfilter.qls
Recording pack reference codeql/python-queries at .../python-queries/0.9.9.
Running the default query suite of codeql/python-queries. In order to run all queries in the query suite, use a clause like this:
- queries: '.'
from: codeql/python-queries
version: 0.9.9 # Optional
Recording pack reference codeql/suite-helpers at .../python-queries/0.9.9/.codeql/libraries/codeql/suite-helpers/0.7.9.
.../python-queries/0.9.9/Expressions/UseofInput.ql
.../python-queries/0.9.9/Security/CVE-2018-1281/BindToAllInterfaces.ql
.../python-queries/0.9.9/Security/CWE-020/IncompleteHostnameRegExp.ql
.../python-queries/0.9.9/Security/CWE-020/IncompleteUrlSubstringSanitization.ql
.../python-queries/0.9.9/Security/CWE-020/OverlyLargeRange.ql
.../python-queries/0.9.9/Security/CWE-022/PathInjection.ql
.../python-queries/0.9.9/Security/CWE-078/CommandInjection.ql
.../python-queries/0.9.9/Security/CWE-079/ReflectedXss.ql
.../python-queries/0.9.9/Security/CWE-089/SqlInjection.ql
.../python-queries/0.9.9/Security/CWE-090/LdapInjection.ql
.../python-queries/0.9.9/Security/CWE-094/CodeInjection.ql
.../python-queries/0.9.9/Security/CWE-116/BadTagFilter.ql
.../python-queries/0.9.9/Security/CWE-209/StackTraceExposure.ql
.../python-queries/0.9.9/Security/CWE-215/FlaskDebug.ql
.../python-queries/0.9.9/Security/CWE-285/PamAuthorization.ql
.../python-queries/0.9.9/Security/CWE-295/MissingHostKeyValidation.ql
.../python-queries/0.9.9/Security/CWE-312/CleartextLogging.ql
.../python-queries/0.9.9/Security/CWE-312/CleartextStorage.ql
.../python-queries/0.9.9/Security/CWE-326/WeakCryptoKey.ql
.../python-queries/0.9.9/Security/CWE-327/BrokenCryptoAlgorithm.ql
.../python-queries/0.9.9/Security/CWE-327/InsecureDefaultProtocol.ql
.../python-queries/0.9.9/Security/CWE-327/InsecureProtocol.ql
.../python-queries/0.9.9/Security/CWE-327/WeakSensitiveDataHashing.ql
.../python-queries/0.9.9/Security/CWE-352/CSRFProtectionDisabled.ql
.../python-queries/0.9.9/Security/CWE-377/InsecureTemporaryFile.ql
.../python-queries/0.9.9/Security/CWE-502/UnsafeDeserialization.ql
.../python-queries/0.9.9/Security/CWE-601/UrlRedirect.ql
.../python-queries/0.9.9/Security/CWE-611/Xxe.ql
.../python-queries/0.9.9/Security/CWE-643/XpathInjection.ql
.../python-queries/0.9.9/Security/CWE-730/PolynomialReDoS.ql
.../python-queries/0.9.9/Security/CWE-730/ReDoS.ql
.../python-queries/0.9.9/Security/CWE-730/RegexInjection.ql
.../python-queries/0.9.9/Security/CWE-776/XmlBomb.ql
.../python-queries/0.9.9/Security/CWE-918/FullServerSideRequestForgery.ql
.../python-queries/0.9.9/Diagnostics/ExtractedFiles.ql
.../python-queries/0.9.9/Diagnostics/ExtractionWarnings.ql
.../python-queries/0.9.9/Summary/LinesOfCode.ql
.../python-queries/0.9.9/Summary/LinesOfUserCode.ql
Also i wondering if the file has content
- qlpack: codeql/python-queries
- include:
id:
- py/missing-docstring
- py/todo-comment
$ codeql resolve queries codeqlfilter.qls
Recording pack reference codeql/python-queries at .../codeql/codeql/qlpacks/codeql/python-queries/0.9.9.
Running the default query suite of codeql/python-queries. In order to run all queries in the query suite, use a clause like this:
- queries: '.'
from: codeql/python-queries
version: 0.9.9 # Optional
Recording pack reference codeql/suite-helpers at .../codeql/qlpacks/codeql/python-queries/0.9.9/.codeql/libraries/codeql/suite-helpers/0.7.9.
WARNING: No queries found in query suite. (..../active/api/codeqlfilter.qls:1,1-1)
It seem codeql not found all queries for python as by default?
@bouillon you could try to change
- qlpack: codeql/python-queries
to
- queries: .
from: codeql/python-queries
to include all Python queries, not just the default ones.
Thanks this include all queries but i get an exception and the output file is not generated. Is there any way to ignore exceptions like this:
-queries/0.9.9/Security/CWE-020-ExternalAPIs/ExternalAPIsUsedWithUntrustedData.ql.
Error was: Unknown kind "Table". [UNSUPPORTED_KIND]
I found this but how to configure it https://github.com/github/codeql/discussions/13839
How to configure and include only compatible queries?
Ok my config
- queries: .
from: codeql/python-queries
- include:
kind: problem
- exclude:
id:
- py/missing-docstring
- py/todo-comment
Any comments are wellcome
This seems like unintended behaviour on our end and I have passed this on to our Python team to have a look to see if there's a Python-specific issue which triggers this error here.
In the meantime, you could try to explicitly exclude table
queries from the query suite:
- queries: .
from: codeql/python-queries
- include:
kind: problem
- exclude:
id:
- py/missing-docstring
- py/todo-comment
- exclude:
kind: table
Let me know if that works for you.
Thank you for support
I am new to CodeQL and have successfully run analyses on our Python codebase locally using the full set of Python queries like this:
codeql database analyze /tmp/codeql_db codeql/python-queries --format csv --output /tmp/codeql-report.csv --download
I would like to exclude certain queries and found the query-filters option:
The documentation for this is found in this issue: https://github.com/github/codeql/issues/7937
However, I'm unsure about where or how to apply the configuration file. The codeql database analyze --help command doesn't accept a configuration file parameter.
I've tried adding this configuration to .github/workflows/codeql-analysis.yml, but it didn't work.
Could you show me what the content of the configuration file should look like and what its name should be?