Xpra-org / xpra

Persistent remote applications for X11; screen sharing for X11, MacOS and MSWindows.
https://xpra.org/
GNU General Public License v2.0
2k stars 171 forks source link

SBOM Software Bill Of Materials #4050

Closed solofoA45 closed 6 days ago

solofoA45 commented 1 year ago

For security and compliance concerns, it would be good to have a list of dependencies for example to assess which security vulnerabilities affect Xpra: https://en.wikipedia.org/wiki/Software_supply_chain

While this is rather clear for linux (RPM) packages, this is less clear for windows packages and HTML5 client packages.

Is there already a way to get these informations?

totaam commented 1 year ago

Here are some starting points and initial thoughts.

This should tie into the Usage : Security documentation since some issues can be mitigated or bypassed using a tight configuration.

The closest thing we have to an overview of the dependencies is here: https://github.com/Xpra-org/xpra/blob/master/docs/Build/Dependencies.md (and includes some pretty diagrams)

MacOS

The MacOS builds are by far the easiest to track since we define and build every single library ourselves. The list can be found by checking out the build repository at the date matching that of the release: https://github.com/Xpra-org/gtk-osx-build There are a few extra packages in there that are only used on MacOS for packaging (ie: xar, etc), those are not shipped.

MS Windows

The full MSYS2 package list on the build system at time of writing has 179 packages:

$ pacman -Qe | wc -l
179

There are also python dependencies which do not have corresponding MINGW packages and are installed via pip: https://github.com/Xpra-org/xpra/blob/15bce9d2ec162ce4984c86aa4279ea2218445194/packaging/MSWindows/SETUP.sh#L38-L42 Then there are also some manual steps: https://github.com/Xpra-org/xpra/blob/15bce9d2ec162ce4984c86aa4279ea2218445194/packaging/MSWindows/SETUP.sh#L46 for:

Packages we do not update as often as others

There can be many reasons why some packages are not updated as regularly as others:

Opaque packages:

3.1

Some issues are magnified in 3.1:

On the whole, I don't think that it is reasonable to expect the 3.1.x to have the same level of maintenance as current versions. CUDA

totaam commented 1 year ago

We should probably split the dependencies into categories - this is probably too many:

totaam commented 11 months ago

The MS Windows dependencies can be recorded in xpra/build_info.py. We can get most of the packages from pacman - and perhaps trim most of the build time dependencies as those aren't very relevant? The python pip dependencies are going to be a pain.

totaam commented 11 months ago

This seems relevant: Understanding the NSA’s latest guidance on managing OSS and SBOMs

totaam commented 10 months ago

Both MacOS and MS Windows builds will now record the libraries and python modules present on the build system when the installer is generated. This will include dependencies we don't really care about: build tools, libraries we don't bundle, etc. But it is safer to record too much than too little, and filtering was hard and would also have required constant fine tuning.

The feature for the html5 client is now tracked here: https://github.com/Xpra-org/xpra-html5/issues/277


Next up:

totaam commented 8 months ago

Another tricky one to handle is pdfium-binaries releases - this release page does show a line that says something like "This version was built with branch chromium/6337 of PDFium". The archive containing the DLL we need also contains a VERSION file:

$ cat pdfium/VERSION 
MAJOR=124
MINOR=0
BUILD=6337
PATCH=0

The easiest way might be to create a "fake" pacman PKGBUILD for it.

totaam commented 8 months ago

The new script that I am working on would flag: liblzma-5.dll as belonging to /mingw64/bin/liblzma-5.dll is owned by mingw-w64-x86_64-xz 5.6.1-2. (current DLL should be fine but it was previously owned by the vulnerable version of xz-utils: 5.6.1-1)

Fixed in: https://github.com/msys2/MSYS2-packages/commit/eb7abbb627d1ccfc5a3a6ca31b98150ee733c366 First vulnerable version was added in: https://github.com/msys2/MSYS2-packages/commit/d153a0914cd45d74acae8a3ade74d0b0d8df5ec6 So any builds between 2024-02-25 and today are shipping the vulnerable library.

On the plus side, the exploit seems to target a specific function in openssh - with glibc, and we don't use openssh by default, and no glibc, and not as a server... So no need to panic.


Good links on the subject:

totaam commented 8 months ago

Forgot another packages missing from MSYS2 that we should contribute upstream: https://github.com/Xpra-org/xpra/blob/79d8e18dc7544d019bef79374b0bdd51c6d42723/packaging/MSWindows/SETUP.sh#L70 Trivial to install: meson build && ninja install.

totaam commented 6 months ago

cyclonedx-python-lib: This Python package provides data models, validators and more, to help you create/render/read CycloneDX documents.

totaam commented 1 month ago

As of 5d21b1b37385223e370cec280b97deba351a6fd1, a default build will add a lot of metadata in xpra/build_info.py:

build = {'user': 'Windows 10 Test', 'on': 'Win10-Build-VM', 'date': '2024-10-20', 'time': '23:10', 'machine': 'AMD64', 'cpu': 'Intel64 Family 6 Model 94 Stepping 3, GenuineIntel', 'bit': '64bit', 'os': 'Microsoft Windows 10 Pro', 'type': '', 'python': '3.11.10', 'cython': '3.0.11', 'compiler': 'gcc (Rev1, Built by MSYS2 project) 14.2.0', 'nvcc': '', 'linker': 'GNU ld (GNU Binutils) 2.43.1'}
libs = {'apr': '1.7.5-1', 'apr-util': '1.6.3-2', 'asciidoc': '10.2.1-2', 'autoconf-wrapper': '20240607-1', ...

(only the first 4 packages are show here since there are literally hundreds of them, as we record all the packages installed on the build system - whether they are used or not, which is impossible to ascertain)

Then also:

BUILD_OPTIONS={'build': True, 'clean': True, 'cuda': False, 'desktop_logon': False, 'docs': False, 'fixups': True, 'html5': False, 'install': True, 'installer': False, 'light': True, 'manual': False, 'msi': False, 'numpy': False, 'openssh': False, 'openssl': False, 'paexec': False, 'putty': False, 'run': True, 'sbom': True, 'service': False, 'sign': True, 'tests': False, 'verbose': False, 'verpatch': True, 'zip': False, 'zip_modules': True}

dll_sbom={'lib/gdk-pixbuf-2.0/2.10.0/loaders/io-wmf.dll': ('mingw-w64-x86_64-libwmf', '0.2.13-1'), 'lib/gdk-pixbuf-2.0/2.10.0/loaders/libpixbufloader-ani.dll': ('mingw-w64-x86_64-gdk-pixbuf2', '2.42.12-3'), ...

Again, far too many to show here. This one is smarter and only includes packages that we actually ship. Only a few DLLs aren't identified yet, which we need to submit upstream: pdfium, libspng, win32com.

This data can be converted to a JSON SBOM using cyclonedx - and maybe this should be a separate build arctifact?

totaam commented 1 month ago

@solofoA45 good news is that the scripting part is mostly done and working well, the bits that still need doing stick out from the script output:

$ python3.11 ./packaging/MSWindows/BUILD.py
* 21:42:25 Collecting version information
    Python 3.11.10 (main, Sep 10 2024, 13:02:13)  [GCC 14.2.0 64 bit (AMD64)]
    Xpra-Light 6.3-r36810
    using 4 cpus
* 21:42:31 locating `signtool`
* 21:42:31 Cleaning output directories and generated files
* 21:42:34 Building Cython modules
* 21:43:27 Generating installation directory
* 21:44:43 Fixups: paths, etc
* 21:44:44 Fixup GStreamer
* 21:44:44 Fixup DLLs
* 21:45:18 Deleting unnecessary DLLs
* 21:45:18 Removing unnecessary Python modules
Warning: glob 'lib/backports' did not match any files!
Warning: glob 'lib/yaml' did not match any files!
* 21:45:18 Removing unnecessary files
* 21:45:21 Removing unnecessary PIL plugins
* 21:45:21 Removing empty directories
* 21:45:22 cuda: False
Warning: glob 'lib/pycuda*' did not match any files!
Warning: glob 'lib/curand*' did not match any files!
* 21:45:22 numpy: False
Warning: 'dist/lib/numpy' does not exist
Warning: glob 'lib/libopenblas*' did not match any files!
Warning: glob 'lib/libgfortran*' did not match any files!
Warning: glob 'lib/libquadmath*' did not match any files!
* 21:45:22 zipping up some Python modules
Warning: glob 'lib/unittest' did not match any files!
Warning: glob 'lib/pynvml' did not match any files!
Warning: glob 'lib/ldap' did not match any files!
Warning: glob 'lib/ldap3' did not match any files!
* 21:45:28 Deleting unnecessary `share/` files
* 21:45:28 Removing empty icon directories
* 21:45:28 Adding EXE manifests
* 21:45:28 Generating gdk pixbuf loaders cache
* 21:45:28 Generating icons and theme cache
* 21:45:28 Recording SBOM
Warning: no package data found for 'lib/libspng-0.dll'
Warning: no package data found for 'pdfium.dll'
Warning: no package data found for 'C:/msys64/mingw64/lib/python3.11/site-packages/aioquic'
Warning: no package data found for 'C:/msys64/mingw64/lib/python3.11/site-packages/pylsqpack'
181     dist
* 21:48:25 Creating ZIP file:
72      Xpra-Light-x86_64_6.3-r36810.zip
* 21:48:44 Creating the installer using InnoSetup
51      Xpra-Light-x86_64_Setup_6.3-r36810.exe
* 21:49:28 Signing EXE
* 21:49:29 Running the new installer

And I think that this should be relatively straightforward to backport this code to older branches:

totaam commented 1 month ago

Here's the beginning of the raw SBOM data for a full build (this is parseable as python source):

# 307 SBOM path entries:
{
'lib/gdk-pixbuf-2.0/2.10.0/loaders/io-wmf.dll': \
    (19975, '39f9911e319826a956a6a858dca4ad5d406e580db8eff3d760245b4b26435540', 'libwmf', '0.2.13-1'),
'lib/gdk-pixbuf-2.0/2.10.0/loaders/libpixbufloader-ani.dll': \
    (26756, '9ed6fb05b84163c24ee2ecf3036b3a23a0f88e2bd29f45bbf1e97cfc079e7216', 'gdk-pixbuf2', '2.42.12-3'),
'lib/gdk-pixbuf-2.0/2.10.0/loaders/libpixbufloader-avif.dll': \
    (25362, '6f99f0d702fc085fadf9d2a9033f0c8349c0b2d73019e5e32a4ffcaa50b3a0bf', 'libavif', '1.1.1-1'),
'lib/gdk-pixbuf-2.0/2.10.0/loaders/libpixbufloader-bmp.dll': \
    (27093, 'bfbba366556397a66c82844a2fea656baa303ab501c112b1139143623a392d13', 'gdk-pixbuf2', '2.42.12-3'), 
...
# 134 'mingw-w64-x86_64-' packages:
('aom', 'atk', 'boost', 'brotli', 'bzip2', 'cairo', 'crypto++', 'curl', 'cyrus-sasl', 'dav1d', 'dbus', 'dbus-glib', 'double-conversion', 'expat', 'faac', 'faad2', 'flac', 'fontconfig', 'freetype', 'fribidi', 'gcc-libgfortran', 'gcc-libs', 'gdk-pixbuf2', 'gettext-runtime', 'glib2', 'gmp', 'gnutls', 'gobject-introspection-runtime', 'graphite2', 'gss', 'gst-plugins-bad', 'gst-plugins-bad-libs', 'gst-plugins-base', 'gst-plugins-good', 'gst-plugins-ugly', 'gstreamer', 'gtk3', 'harfbuzz', 'highway', 'icu', 'jbigkit', 'lame', 'lcms2', 'lerc', 'libavif', 'libb2', 'libcroco', 'libdatrie', 'libdeflate', 'libepoxy', 'libffi', 'libgcrypt', 'libgpg-error', 'libheif', 'libiconv', 'libidn', 'libidn2', 'libimagequant', 'libjpeg-turbo', 'libjxl', 'libogg', 'libpng', 'libpsl', 'libraqm', 'librsvg', 'libsodium', 'libsoup', 'libssh2', 'libtasn1', 'libthai', 'libtiff', 'libunistring', 'libvorbis', 'libvpx', 'libwebp', 'libwinpthread-git', 'libwmf', 'libx264', 'libxml2', 'libyaml', 'libyuv', 'lz4', 'md4c', 'mpdecimal', 'mpg123', 'nettle', 'nghttp2', 'nghttp3', 'openblas', 'openh264', 'openjpeg2', 'openjph', 'openldap', 'openssl', 'opus', 'orc', 'p11-kit', 'pango', 'pcre2', 'pixman', 'python', 'python-bcrypt', 'python-cairo', 'python-cffi', 'python-charset-normalizer', 'python-cryptography', 'python-importlib-metadata', 'python-importlib_resources', 'python-ldap', 'python-netifaces', 'python-numpy', 'python-oauthlib', 'python-pillow', 'python-py-cpuinfo', 'python-pycryptodome', 'python-pycryptodomex', 'python-pynacl', 'python-pyopengl-accelerate', 'python-pyparsing', 'python-pywin32', 'python-watchdog', 'python-winkerberos', 'python-zeroconf', 'qrencode', 'qt6-base', 'rav1e', 'shishi', 'sqlite3', 'svt-av1', 'wavpack', 'xxhash', 'xz', 'zlib', 'zstd')
totaam commented 1 month ago

Some of the missing MSYS2 packages have been submitted upstream:

This one I will probably keep in my own repo as I don't think that will be acceptable for upstream:

Which only leaves CUDA. It makes sense to special case it since we already have a add_cuda function and cuda switch. They ship their own metadata file which we'll need to parse:

$ cat /z/version.json 
{
   "cuda" : {
      "name" : "CUDA SDK",
      "version" : "12.6.0"
   },
...
totaam commented 1 month ago

The MSYS2 packages have been merged upstream and the CUDA information is also recorded now using custom code: 8926b62f435548f05cb78df061c23bf54f5b67b3.


Unfortunately, the call order was wrong and zipping the python modules was hiding some packages from the SBOM code. So now after re-ordering these two steps, we see more missing modules:

* 17:30:35 Recording SBOM
Warning: no package data found for 'C:/msys64/mingw64/lib/python3.11/site-packages/browser_cookie3'
Warning: unknown source for filename 'browser_cookie3', tried [
    'C:/msys64/mingw64/lib/python3.11',
    'C:/msys64/mingw64/lib/python3.11/lib-dynload',
    'C:/msys64/mingw64/lib/python3.11/site-packages',
    'C:/msys64/mingw64/lib/python3.11/site-packages/win32',
    'C:/msys64/mingw64/lib/python3.11/site-packages/pywin32_system32'
]
Warning: unknown source for filename 'pkcs11', tried [...
Warning: no package data found for 'C:/msys64/mingw64/lib/python3.11/site-packages/pyaes'
Warning: unknown source for filename 'pyaes', tried [...
Warning: no package data found for 'C:/msys64/mingw64/lib/python3.11/site-packages/pycuda'
Warning: unknown source for filename 'pycuda', tried [...
Warning: no package data found for 'C:/msys64/mingw64/lib/python3.11/site-packages/pyvda'
Warning: unknown source for filename 'pyvda', tried [...
* 17:35:16 zipping up some Python modules

Links:

totaam commented 1 month ago

The commits above adds some data for macos builds. This data is much less useful because jhbuild doesn't expose any package file tracking so we can resolve specific files in our bundle to the package that installed them. At least the packages data lists which version of each package was installed. (and we skip the ones only used for packaging)

This will do for now. The remaining warnings about unpackaged python dependencies will serve as a reminder that this can be improved further.

I would have liked to also generate the SBOM in a more standardized format, perhaps using cyclonedx-python but the lack of tools made this too difficult.

totaam commented 1 month ago

Example usage:

totaam commented 3 weeks ago

We now export a fairly comprehensive sbom using cyclonedx. (it must be installed separately in a Python-for-MSWindows installation since cyclonedx-py is not compatible with MSYS2)

The problem is that this now flags some missing package information via dependencies:

Warning: 'mingw-w64-x86_64-lzo2' not found, dependency of cairo
Warning: 'mingw-w64-x86_64-c-ares' not found, dependency of curl
Warning: 'mingw-w64-x86_64-ca-certificates' not found, dependency of curl
Warning: 'mingw-w64-x86_64-libwinpthread' not found, dependency of gcc-libs
Warning: 'mingw-w64-x86_64-nettle>=3.6' not found, dependency of gnutls
Warning: 'mingw-w64-x86_64-p11-kit>=0.23.1' not found, dependency of gnutls
Warning: 'mingw-w64-x86_64-adwaita-icon-theme' not found, dependency of gtk3
Warning: 'mingw-w64-x86_64-json-glib' not found, dependency of gtk3
Warning: 'mingw-w64-x86_64-shared-mime-info' not found, dependency of gtk3
Warning: 'mingw-w64-x86_64-gtk-update-icon-cache' not found, dependency of gtk3
Warning: 'mingw-w64-x86_64-aom' not found, dependency of libavif
Warning: 'mingw-w64-x86_64-dav1d' not found, dependency of libavif
Warning: 'mingw-w64-x86_64-rav1e' not found, dependency of libavif
Warning: 'mingw-w64-x86_64-svt-av1' not found, dependency of libavif
Warning: 'mingw-w64-x86_64-libsharpyuv' not found, dependency of libavif
Warning: 'mingw-w64-x86_64-aom' not found, dependency of libheif
Warning: 'mingw-w64-x86_64-dav1d' not found, dependency of libheif
Warning: 'mingw-w64-x86_64-kvazaar' not found, dependency of libheif
Warning: 'mingw-w64-x86_64-libde265' not found, dependency of libheif
Warning: 'mingw-w64-x86_64-libsharpyuv' not found, dependency of libheif
Warning: 'mingw-w64-x86_64-rav1e' not found, dependency of libheif
Warning: 'mingw-w64-x86_64-svt-av1' not found, dependency of libheif
Warning: 'mingw-w64-x86_64-x265' not found, dependency of libheif
Warning: 'mingw-w64-x86_64-giflib' not found, dependency of libjxl
Warning: 'mingw-w64-x86_64-libwinpthread' not found, dependency of libjxl
Warning: 'mingw-w64-x86_64-openexr' not found, dependency of libjxl
Warning: 'mingw-w64-x86_64-glib-networking' not found, dependency of libsoup
Warning: 'mingw-w64-x86_64-sqlite3' not found, dependency of libsoup
Warning: 'mingw-w64-x86_64-giflib' not found, dependency of libwebp
Warning: 'mingw-w64-x86_64-libjpeg' not found, dependency of libwmf
Warning: 'mingw-w64-x86_64-libjpeg' not found, dependency of libyuv
Warning: 'mingw-w64-x86_64-wineditline' not found, dependency of pcre2
Warning: 'mingw-w64-x86_64-mpdecimal' not found, dependency of python
Warning: 'mingw-w64-x86_64-ncurses' not found, dependency of python
Warning: 'mingw-w64-x86_64-sqlite3' not found, dependency of python
Warning: 'mingw-w64-x86_64-tzdata' not found, dependency of python
Warning: 'mingw-w64-x86_64-python-decorator' not found, dependency of python-gssapi
Warning: 'mingw-w64-x86_64-gss' not found, dependency of python-gssapi
Warning: 'mingw-w64-x86_64-python-jaraco.classes' not found, dependency of python-keyring
Warning: 'mingw-w64-x86_64-python-jaraco.context' not found, dependency of python-keyring
Warning: 'mingw-w64-x86_64-python-jaraco.functools' not found, dependency of python-keyring
Warning: 'mingw-w64-x86_64-python-pywin32-ctypes' not found, dependency of python-keyring
Warning: 'mingw-w64-x86_64-python-six' not found, dependency of python-paramiko
Warning: 'mingw-w64-x86_64-libjpeg' not found, dependency of python-pillow
Warning: 'mingw-w64-x86_64-python-six' not found, dependency of python-pyu2f

We already skip most of the ones we don't need: https://github.com/Xpra-org/xpra/blob/0dfe55ed50404565e53d4fab4a47ea4d5295c4ec/packaging/MSWindows/cyclonedx_sbom.py#L73-L77 More should be added to this list. Then others should be recorded properly (pyopengl, libjpeg, etc). Others only provide data files, but we may still want to track these..

totaam commented 2 weeks ago

Remaining warnings for light builds during the cyclonedx export step that still need to be investigated:

Warning: 'mingw-w64-x86_64-c-ares' not found, dependency of curl
Warning: 'mingw-w64-x86_64-ca-certificates' not found, dependency of curl
Warning: 'mingw-w64-x86_64-nettle>=3.6' not found, dependency of gnutls
Warning: 'mingw-w64-x86_64-p11-kit>=0.23.1' not found, dependency of gnutls
Warning: 'mingw-w64-x86_64-glib-networking' not found, dependency of libsoup
Warning: 'mingw-w64-x86_64-sqlite3' not found, dependency of libsoup
Warning: 'mingw-w64-x86_64-libsystre' not found, dependency of ncurses
Warning: 'mingw-w64-x86_64-wineditline' not found, dependency of pcre2
Warning: 'mingw-w64-x86_64-mpdecimal' not found, dependency of python
Warning: 'mingw-w64-x86_64-sqlite3' not found, dependency of python
Warning: 'mingw-w64-x86_64-python-decorator' not found, dependency of python-gssapi
Warning: 'mingw-w64-x86_64-gss' not found, dependency of python-gssapi
totaam commented 2 weeks ago

With the latest dependency updates and tweaks, only a few corner cases remain for the "light" builds:

totaam commented 2 weeks ago

To make things even more "interesting", MSYS2 just switched from Python 3.11 to Python 3.12. So the 6.3-r36873 beta builds are still using Python 3.11.10, but any later builds will be based on Python 3.12.x.

Good news is that the upgrade just worked. All that was needed afterwards - as expected:

totaam commented 1 week ago

The export-sbom step of the new build script can be executed separately:

packaging/MSWindows/BUILD.py export-sbom

It requires:

totaam commented 6 days ago

There is now a link from the download page to the corresponding sbom files. (only for the beta releases for now, others will be added later)

totaam commented 6 days ago

Caveats:

Closing.

totaam commented 11 hours ago

With the latest fixes, the only missing piece is a PKGBUILD for pycuda.