Closed RobertBerger closed 4 years ago
I guess there is a double inherit of sca - one from local.conf and one from inherit ${@bb.utils.contains('DISTRO_FEATURES', 'sca', 'sca', '', d)}
- this is something bitbake can't resolve.
So for me removing INHERIT += "sca"
from local.conf should do the trick, as the magic begins at inherit ${@bb.utils.contains('DISTRO_FEATURES', 'sca', 'sca', '', d)}
the rest is just configuration.
It should work this way - please do me the favor and try it.
BTW interesting setup you created - wasn't my intention to use it like that, but I'm glad it could be used in this way.
If I remove INHERIT += "sca"
cvecheck
is not run on all the recipes included in the image but only on the few recipes which interited sca.
BTW interesting setup you created - wasn't my intention to use it like that, but I'm glad it could be used in this way.
Well I want to start with certain checks of code I'm in control of to start with since finding shit loads of warnings in open source projects will not be fixed to quickly.
At the moment I have a build running where I made a .bbappend
to an image recipe which inherits sca and my sca-global-checks
, but this didn't trigger cvecheck
.
bitbake core-image-minimal -e | grep SCA_ENABLED_MODULES
# $SCA_ENABLED_MODULES
# "${SCA_ENABLED_MODULES_${SCA_MODE_UPPER}}"
SCA_ENABLED_MODULES=" cvecheck "
# $SCA_ENABLED_MODULES_IMAGE [2 operations]
SCA_ENABLED_MODULES_IMAGE=" cvecheck "
# $SCA_ENABLED_MODULES_RECIPE
SCA_ENABLED_MODULES_RECIPE=" cvecheck "
for item in intersect_lists(d, d.getVar("SCA_ENABLED_MODULES"), d.getVar("SCA_AVAILABLE_MODULES")):
if "bitbake" in d.getVar("SCA_ENABLED_MODULES").split(" "):
but
bitbake busybox -e | grep SCA_ENABLED_MODULES
because globally INHERIT += "sca"
is missing
Hmm I think I need to switch project, since some mender class seems to
bitbake sca-hello-world -e | grep IMAGE_INSTALL
# $IMAGE_INSTALL [2 operations]
# [doc] "Specifies the packages to install into an image. Image recipes set IMAGE_INSTALL to specify the packages to install into an image through image.bbclass."
IMAGE_INSTALL=" mender"
which confuses sca.
We are back in business with another project
bitbake sca-hello-world -e | grep IMAGE_INSTALL
# $IMAGE_INSTALL
# [doc] "Specifies the packages to install into an image. Image recipes set IMAGE_INSTALL to specify the packages to install into an image through image.bbclass."
# [doc] "When set to '1', no recommended packages will be installed. Realize that some recommended packages might be required for certain system functionality, such as kernel-modules. It is up to the user to add packages to IMAGE_INSTALL as needed."
# [doc] "List of the packages to be installed into the image. The variable is generally not user-defined and uses IMAGE_INSTALL as part of the list."
bitbake sca-hello-world -e | grep SCA_ENABLED_MODULES
# $SCA_ENABLED_MODULES_IMAGE [4 operations]
SCA_ENABLED_MODULES_IMAGE=" bitbake oelint "
# $SCA_ENABLED_MODULES_RECIPE [4 operations]
SCA_ENABLED_MODULES_RECIPE=" bitbake oelint cspell gcc cppcheck cpplint cqmetrics "
I just pushed a change which addresses the IMAGE_INSTALL assignment within a recipe - Please give me some feedback if it solve one on the issue you have with integration of SCA
looks much better now:
bitbake sca-hello-world -e | grep IMAGE_INSTALL
# $IMAGE_INSTALL [2 operations]
# [doc] "Specifies the packages to install into an image. Image recipes set IMAGE_INSTALL to specify the packages to install into an image through image.bbclass."
IMAGE_INSTALL=" mender"
# [doc] "When set to '1', no recommended packages will be installed. Realize that some recommended packages might be required for certain system functionality, such as kernel-modules. It is up to the user to add packages to IMAGE_INSTALL as needed."
# [doc] "List of the packages to be installed into the image. The variable is generally not user-defined and uses IMAGE_INSTALL as part of the list."
bitbake sca-hello-world -e | grep SCA_ENABLED_MODULES
# $SCA_ENABLED_MODULES_IMAGE [4 operations]
SCA_ENABLED_MODULES_IMAGE=" bitbake oelint "
# $SCA_ENABLED_MODULES_RECIPE [4 operations]
SCA_ENABLED_MODULES_RECIPE=" bitbake oelint cspell gcc cppcheck cpplint cqmetrics "
There is still the issue with globally enabling things plus special things per recipe.
This is what I currently have in my local.conf
# --> SCA stuff
# DISTRO_FEATURES_append = " sca"
# let's not force sca by default, but it can be turned on here
#SCA_FORCE_RUN = "1"
SCA_CLEAN_BEFORE_EXPORT = "1"
### Summary: Insert this into your local.conf or distro.conf ####
#SCA_AUTO_INH_ON_IMAGE = "1"
#SCA_AUTO_INH_ON_RECIPE = "1"
SCA_AUTO_LICENSE_FILTER = ".*"
# SCA_AVAILABLE_MODULES: List of all available modules, use to globally enable/disable modules
# bitbake -e | grep ^SCA_AVAILABLE_MODULES=
# SCA_AVAILABLE_MODULES = " ansible ansiblelint bandit bashate bitbake checkbashism clang cppcheck cpplint cqmetrics cspell cvecheck dennis detectsecrets
# eslint flint gcc gixy htmlhint jsonlint kconfighard oclint oelint proselint pyfindinjection pylint pysymcheck pytype
# radon rats ropgadget score shellcheck standard stylelint systemdlint textlint tlv tscancode xmllint yamllint zrd"
#
# SCA_ENABLED_MODULES: The analysis modules to be activated - different lists for images and recipes
#
# on image:
# bitbake app-container-image-python3-data-collector -e | grep ^SCA_ENABLED_MODULES=
#
# SCA_ENABLED_MODULES=" ansible ansiblelint bandit bashate bitbake checkbashism detectsecrets eslint gixy htmlhint jsonlint oelint proselint
# pyfindinjection pylint shellcheck standard stylelint systemdlint tlv xmllint yamllint"
#
# on recipe:
# bitbake python3-tcpclientserver -e | grep ^SCA_ENABLED_MODULES=
# SCA_ENABLED_MODULES=" ansiblelint bandit bashate bitbake checkbashism clang cppcheck cpplint cqmetrics cspell cvecheck dennis detectsecrets
# eslint flint gcc htmlhint jsonlint kconfighard oclint oelint proselint pyfindinjection pylint pysymcheck pytype
# radon rats ropgadget score shellcheck standard stylelint textlint tlv tscancode xmllint yamllint zrd"
#
# empty the defaults
SCA_ENABLED_MODULES_IMAGE = ""
SCA_ENABLED_MODULES_RECIPE = ""
# inherit sca globally
INHERIT += "sca"
# turn on global checks
INHERIT += "sca-global-checks"
SCA_ENABLE_IMAGE_SUMMARY = "1"
SCA_EXPORT_FINDING_SRC = "1"
SCA_WARNING_LEVEL = "warning"
SCA_STD_PYTHON_INTERPRETER = "python3"
# <-- SCA stuff
and it leads to the dependency loops
.
I would not bet on it, but it looks like it would happen also if I only do INHERIT += "sca" in local.conf
since I took out all the inherit sca
stuff form my recipes.
files=$(grep -rl -w "inherit \${@bb.utils.contains('DISTRO_FEATURES', 'sca', 'sca', '', d)}" . | cut -f 1 -d ':' | sort | uniq) && echo $files
./meta-desire/recipes-core/images/app-container-image-python3-data-collector.bb ./meta-desire/recipes-core/images/app-container-image-python3-nmap-srv.bb ./meta-desire/recipes-desire/python-iot/python3-localdatacollector_git.bb ./meta-desire/recipes-desire/python-iot/python3-mastermind_git.bb ./meta-desire/recipes-desire/python-iot/python3-messageqclient_git.bb ./meta-desire/recipes-desire/python-iot/python3-mqttbrokerclient_git.bb ./meta-desire/recipes-desire/python-iot/python3-nmapwrapper_git.bb ./meta-desire/recipes-desire/python-iot/python3-tcpclientserver_git.bb ./meta-resy/recipes-connectivity/softether/softether_git.inc ./meta-resy/recipes-connectivity/softether/softether-hamcorebuilder-native_git.inc ./meta-resy/recipes-example/sca-hello-world/sca-hello-world_1.0.0.bb
I don't include meta-desire
in by bblayers.conf
and commented out the inherit
lines in meta-resy
.
Could it be that the problems come from the fact that me and you have the same recipes in our layers?
This is the bitbake
error log.
I see I see, the issue is that sca include into -native-recipes. In this case into python3-native. I'll fix this
Just fixed the issue in master - Please give it a try
@RobertBerger is this fixed now?
There hasn't been any progress for ~2 month, so I assume that the reported issue is fixed. If there is anything to add, please reopen
I think I managed to enable for the recipes I like the tools I like as well. e.g. for C checks:
So far so good. This seems to work. So in my recipes I do the following:
Now I would like to have
cvecheck
run on all recipes when I build an image.Therefor I thought I just need to add to
local.conf
but now bitbake complains about circular dependencies.
How would you suggest I do something like this?
I would like to run certain tools (like for the moment
cvecheck
) on every recipe used in an image, other tools on C projects and other tools on Python projects.I guess my logic to do that is a bit broken ;)