search

google / osv-scanner

Vulnerability scanner written in Go which uses the data provided by https://osv.dev
https://google.github.io/osv-scanner/
Apache License 2.0
6.04k stars 337 forks source link

[WebKit vendored code] osv-scanner fails to identify multiple third-party projects #803

Open ddkilzer opened 4 months ago

ddkilzer commented 4 months ago

Summary:

osv-scanner fails to identify multiple third-party projects in the WebKit project while scanning for vendored code dependencies.

Steps to Reproduce:

  1. Check out WebKit (at commit https://github.com/WebKit/WebKit/commit/fda388552a877f757aa8216c8d116937fe8651f2): 
    git clone https://github.com/WebKit/WebKit.git WebKit.git
  2. Run osv-scanner (at commit 85563d901bec48bbe8db1242f083c42d42353ace): 
    go run ./cmd/osv-scanner/main.go -r WebKit.git/Source/ThirdParty

Expected Results:

osv-scanner identifies multiple third-party projects as vendored code dependencies.

Actual Results:

osv-scanner fails to identify multiple third-party projects as vendored code dependencies.

I'm not sure if all of these are tracked by osv-scanner, but at least some of them are since they're fuzzed by oss-fuzz.

Scanning dir WebKit.git/Source/ThirdParty
Scanning directory for vendored libs: WebKit.git/Source/ThirdParty
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/ANGLE
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/capstone
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/d3flamegraphjs
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/d3js
[...]
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/libwebrtc
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/pdfjs
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/qunit
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/skia
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/xdgmime
Scanning directory for vendored libs: WebKit.git/Source/ThirdParty/ANGLE/src/common/third_party
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/ANGLE/src/common/third_party/xxhash
Scanning directory for vendored libs: WebKit.git/Source/ThirdParty/ANGLE/src/libANGLE/renderer/vulkan/shaders/src/third_party
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/ANGLE/src/libANGLE/renderer/vulkan/shaders/src/third_party/etc_decoder
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/ANGLE/src/libANGLE/renderer/vulkan/shaders/src/third_party/ffx_spd
Scanning directory for vendored libs: WebKit.git/Source/ThirdParty/ANGLE/src/tests/perf_tests/third_party
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/ANGLE/src/tests/perf_tests/third_party/perf
Scanning directory for vendored libs: WebKit.git/Source/ThirdParty/ANGLE/src/tests/test_utils/third_party
Scanning directory for vendored libs: WebKit.git/Source/ThirdParty/ANGLE/src/third_party
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/ANGLE/src/third_party/ceval
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/ANGLE/src/third_party/khronos
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/ANGLE/src/third_party/libXNVCtrl
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/ANGLE/src/third_party/volk
Scanning directory for vendored libs: WebKit.git/Source/ThirdParty/ANGLE/third_party
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/ANGLE/third_party/EGL-Registry
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/ANGLE/third_party/OpenCL-Docs
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/ANGLE/third_party/OpenCL-ICD-Loader
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/ANGLE/third_party/OpenGL-Registry
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/ANGLE/third_party/VK-GL-CTS
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/ANGLE/third_party/android_system_sdk
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/ANGLE/third_party/astc-encoder
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/ANGLE/third_party/bazel
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/ANGLE/third_party/clspv
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/ANGLE/third_party/colorama
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/ANGLE/third_party/cpu_features
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/ANGLE/third_party/flatbuffers
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/ANGLE/third_party/glmark2
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/ANGLE/third_party/jdk
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/ANGLE/third_party/libpng
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/ANGLE/third_party/llvm
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/ANGLE/third_party/logdog
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/ANGLE/third_party/mesa
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/ANGLE/third_party/minigbm
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/ANGLE/third_party/proguard
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/ANGLE/third_party/r8
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/ANGLE/third_party/rapidjson
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/ANGLE/third_party/renderdoc
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/ANGLE/third_party/turbine
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/ANGLE/third_party/zlib
Scanning directory for vendored libs: WebKit.git/Source/ThirdParty/ANGLE/tools/flex-bison/third_party
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/ANGLE/tools/flex-bison/third_party/m4sugar
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/ANGLE/tools/flex-bison/third_party/skeletons
Scanning directory for vendored libs: WebKit.git/Source/ThirdParty/ANGLE/util/android/third_party
Scanning directory for vendored libs: WebKit.git/Source/ThirdParty/ANGLE/util/windows/third_party
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/ANGLE/util/windows/third_party/StackWalker
Scanning directory for vendored libs: WebKit.git/Source/ThirdParty/libwebrtc/Source/third_party
[...]
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/libwebrtc/Source/third_party/crc32c
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/libwebrtc/Source/third_party/json
[...]
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/libwebrtc/Source/third_party/libyuv
[...]
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/libwebrtc/Source/third_party/pffft
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/libwebrtc/Source/third_party/rnnoise
[...]
Scanning directory for vendored libs: WebKit.git/Source/ThirdParty/libwebrtc/Source/third_party/boringssl/src/third_party
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/libwebrtc/Source/third_party/boringssl/src/third_party/fiat
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/libwebrtc/Source/third_party/boringssl/src/third_party/googletest
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/libwebrtc/Source/third_party/boringssl/src/third_party/wycheproof_testvectors
Scanning directory for vendored libs: WebKit.git/Source/ThirdParty/libwebrtc/Source/third_party/libaom/source/libaom/third_party
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/libwebrtc/Source/third_party/libaom/source/libaom/third_party/SVT-AV1
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/libwebrtc/Source/third_party/libaom/source/libaom/third_party/fastfeat
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/libwebrtc/Source/third_party/libaom/source/libaom/third_party/vector
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/libwebrtc/Source/third_party/libaom/source/libaom/third_party/x86inc
Scanning directory for vendored libs: WebKit.git/Source/ThirdParty/libwebrtc/Source/third_party/libvpx/source/libvpx/third_party
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/libwebrtc/Source/third_party/libvpx/source/libvpx/third_party/googletest
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/libwebrtc/Source/third_party/libvpx/source/libvpx/third_party/libwebm
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/libwebrtc/Source/third_party/libvpx/source/libvpx/third_party/libyuv
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/libwebrtc/Source/third_party/libvpx/source/libvpx/third_party/x86inc
Scanning directory for vendored libs: WebKit.git/Source/ThirdParty/libwebrtc/Source/webrtc/common_audio/third_party
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/libwebrtc/Source/webrtc/common_audio/third_party/ooura
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/libwebrtc/Source/webrtc/common_audio/third_party/spl_sqrt_floor
Scanning directory for vendored libs: WebKit.git/Source/ThirdParty/libwebrtc/Source/webrtc/examples/androidapp/third_party
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/libwebrtc/Source/webrtc/examples/androidapp/third_party/autobanh
Scanning directory for vendored libs: WebKit.git/Source/ThirdParty/libwebrtc/Source/webrtc/examples/androidtests/third_party
Scanning directory for vendored libs: WebKit.git/Source/ThirdParty/libwebrtc/Source/webrtc/examples/objc/AppRTCMobile/third_party
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/libwebrtc/Source/webrtc/examples/objc/AppRTCMobile/third_party/SocketRocket
Scanning directory for vendored libs: WebKit.git/Source/ThirdParty/libwebrtc/Source/webrtc/modules/third_party
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/libwebrtc/Source/webrtc/modules/third_party/fft
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/libwebrtc/Source/webrtc/modules/third_party/g711
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/libwebrtc/Source/webrtc/modules/third_party/g722
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/libwebrtc/Source/webrtc/modules/third_party/portaudio
Scanning directory for vendored libs: WebKit.git/Source/ThirdParty/libwebrtc/Source/webrtc/rtc_base/third_party
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/libwebrtc/Source/webrtc/rtc_base/third_party/base64
Scanning potential vendored dir: WebKit.git/Source/ThirdParty/libwebrtc/Source/webrtc/rtc_base/third_party/sigslot
Scanning directory for vendored libs: WebKit.git/Source/ThirdParty/libwebrtc/Source/webrtc/tools_webrtc/libs
[...]

Notes:

osv-scanner ends on a parsing error:

[...]
Failed to run code analysis (govulncheck) on 'WebKit.git/Source/ThirdParty/libwebrtc/Source/third_party/boringssl/src/go.mod' because govulncheck: loading packages: 
There are errors with the provided package patterns:

-: break-kat.go: parsing //go:build line: unexpected end of expression

For details on package patterns, see https://pkg.go.dev/cmd/go#hdr-Package_lists_and_patterns.

(the Go toolchain is required)
[...]
ddkilzer commented 4 months ago

Note that ANGLE and webrtc projects are covered by Issue #802.

I filed this to cover the many, smaller vendored projects in WebKit.

Also, the Notes section of Issue #801 mentions the two partial googletest projects in the output above.