`check` does not give any useful output, all components are skipped (IDFGH-11259)

[andyn-ff]

I am testing this tool in the IDF container shared to Docker Hub by Espressif, v5.1.1. I have verified that the two required commits detailed in the README are ancestors of the v5.1.1 tag so I don't think there is any need to run git cherry-pick.

When I use esp-idf-sbom to create a spdx SBOM and then check it against the vulnerability database, all the components are skipped so I do not get useful information on vulnerabilities.

In addition, there is significantly more helpful licence information in spdx file produced by version 0.5.0 of esp-idf-sbom vs later versions.

Test code:

CONTAINER_ID=$(docker run --rm -it --detach espressif/idf:v5.1.1)
TIMESTAMP=$(date +"%Y%m%d-%H%M")
docker exec -i $CONTAINER_ID bash -c '
cd /opt/esp/idf/examples/get-started/hello_world/
git describe
git merge-base --is-ancestor 0f781c718c8548cd2b0e41a30e1814f1c6ed93a2 HEAD
echo $?
git merge-base --is-ancestor 03162bb276d4155760e8aa839020f0587f5ef599 HEAD
echo $?
. /opt/esp/idf/export.sh
idf.py build
pip install esp-idf-sbom==0.5.0
esp-idf-sbom create -o hello-world.spdx build/project_description.json
esp-idf-sbom check --format json hello-world.spdx > hello-world-check.json
'
docker cp $CONTAINER_ID:/opt/esp/idf/examples/get-started/hello_world/hello-world.spdx hello-world_$TIMESTAMP.spdx
docker cp $CONTAINER_ID:/opt/esp/idf/examples/get-started/hello_world/hello-world-check.json hello-world-check_$TIMESTAMP.json

Resulting files for versions 0.5.0, 0.6.0, and 0.8.0 (latest) of esp-idf-sbom attached: sample_output.zip

Please can you help me understand if I am doing something wrong here?

[fhrbata]

Hello @andyn-ff , thank you for looking into esp-idf-sbom and your feedback.

When I use esp-idf-sbom to create a spdx SBOM and then check it against the vulnerability database, all the components are skipped so I do not get useful information on vulnerabilities.

esp-idf-sbom checker uses CPE to check for vulnerabilities. If some component does not have sbom.yml manifest or it is missing the cpe entry in it, this component will not be checked(SKIPPED). We are constantly adding sbom.yml files with all necessary info, but currently not all components have it. We already have made a progress, so there should be more and more components covered/checked in the future.

Also by default only components, which are actually linked into the final binary image, are checked. For example mbedtls will not be checked if it's not linked. You can use esp-idf-sbom check --check-all-packages hello-world.spdx to check all components, even if they are not linked.

It should be possible to scan the generated spdx file with some 3rd party tooling, like cve-bin-tool. For example $ cve-bin-tool --sbom spdx --sbom-file hello-world.spdx.

You can also check the manifest files directly without generating the SBOM SPDX file with $ esp-idf-sbom manifest check. This will scan all components in current directory which have the CPE information available. You can use single manifest file or mix of manifests files and directories.

In addition, there is significantly more helpful licence information in spdx file produced by version 0.5.0 of esp-idf-sbom vs later versions.

Yes, we changed the default behavior. Previously the license and copyright info was added by default and this could be disabled with --no-file-tags. Since the generated spdx could be pretty big, we reverted the logic. Now only the minimal spdx file is generated by default, which is enough for the vulnerabilities scanning. You can add the license and copyright info into the generated spdx file with the --file-tags option. There are few options which define what/how will be included in the generated spdx file. We are still trying to figure out the best combination.

Please let me know if this answers your questions/concerns or if I've missed something. Any feedback is very much appreciated!

Thank you very much

[andyn-ff]

Thank you, that makes sense, I understand the behaviour now.

I will try cve-bin-tool.

I see there are some more sbom.yaml files added to components (e.g. FreeRTOS) on the master branch of the esp-idf repository, so I assume these might be included in IDF v5.2. Will you aim to have a manifest file in each component containing third-party code by a particular milestone, or will this be an incremental process over some months/years?

[fhrbata]

Hello @andyn-ff , currently we should have sbom manifest with CPE info for freertos, lwip, mbedtls, newlib, micro-ecc, cJSON and protobuf-c in esp-idf or submodules it uses and libcoap, libexpat, ftm, freetype, libpng, nghttp2, zlib, catch2 in idf-extra-components. WRT esp-idf these are also backported to older esp-idf releases and as you pointed out, they will be available in next minor releases.

Will you aim to have a manifest file in each component containing third-party code by a particular milestone, or will this be an incremental process over some months/years?

IIUC we haven't set any milestone for this, but we are actively tracking and adding manifests for 3rd party components/libs, specially the highly exposed ones, which have CPE available in NVD. So this is incremental process.

@mahavirj please correct me if I've missed something.

Thank you

[mahavirj]

Hello @andyn-ff,

What @fhrbata mentioned is absolutely correct. Just to add, the effort of bringing all external components under SBOM coverage definitely won't take years. We are actively working on it and hope to converge it soon. We also plan to backport the changes to actively maintained release branches.

HTH!

Thanks.

[dobairoland]

Can this be closed? It seems that we cannot do anything about this on the esp-idf-sbom side.

[fhrbata]

@dobairoland yes, I believe so. @andyn-ff hopefully this answers your question. Anyway feel free to re-open or file a new issue if necessary. Thank you all!

[fhrbata]

@dobairoland it looks like I don't have rights to close this. Would you mind closing it? Thank you