MacOS Build packaging failing due to checksum for Packages.dmg no longer matching expected checksum.

FiniteSingularity commented 11 months ago

Operating System Info

Windows 11

Other OS

No response

OBS Studio Version

30.0.0-beta2

OBS Studio Version (Other)

N/A

OBS Studio Log URL

N/A

OBS Studio Crash Log URL

N/A

Expected Behavior

App should build and package properly, as the Packages.dmg checksum should be consistent.

Current Behavior

This is a potential security issue with the current plugin template. The downloaded dmg of the app used to package the MacOS version of the plugin (Packages.dmg) does not have a checksum that matches the expected checksum, and according to the Packages developer website and GitHub, no update has been posted since Feb of 2022. I was able to find a WebArchive (wayback machine) version of the file from July of this summer.

Current Packages.dmg Download: http://s.sudre.free.fr/Software/files/Packages.dmg Wayback Machine: https://web.archive.org/web/20230727054218/http://s.sudre.free.fr/Software/files/Packages.dmg

Here are the results when looking at file size and checksum:

~/Downloads 
❯ ls -la Packages-*.dmg 
-rw-r--r--@ 1 mark  staff  8398465 Sep 24 13:09 Packages-Current.dmg
-rw-r--r--@ 1 mark  staff  8398202 Sep 24 13:22 Packages-Wayback.dmg

~/Downloads 
❯ sha256sum Packages-*.dmg       
9d9a73a64317ea6697a380014d2e5c8c8188b59d5fb8ce8872e56cec06cd78e8  Packages-Current.dmg
6afdd25386295974dad8f078b8f1e41cabebd08e72d970bf92f707c7e48b16c9  Packages-Wayback.dmg

Note- the Wayback version of the image has a checksum that matches the current checksum in the macOS package GH action script.

Something changed the file size and thus the checksum of the Packages.dmg file on the developer's server last night (I know it happened in the last 24 hours as I packaged up a plugin yesterday without issue). However, the version of the Packages app which is to be installed in both cases is the same: v1.2.10.

Honestly, the more I dig into the Packages app, the more concerned I get about using it. The fact that the downloads page (and the maintainer website) are not even https is a red flag to me, especially when checksums are changing. And as this is a packaging utility that generates .pkg files for plug-in users to install with (potentially with elevated permissions) its not something to take lightly.

Edit- I just checked the version of the webserver that is serving up the files, the Packages.dmg file was modified yesterday, and the Apache server being used was built in 2019:

❯ curl --head http://s.sudre.free.fr/Software/files/Packages.dmg
HTTP/1.1 200 OK
Date: Sun, 24 Sep 2023 21:06:10 GMT
Server: Apache/ProXad [Jan 23 2019 20:05:46]
Last-Modified: Sat, 23 Sep 2023 20:43:31 GMT
ETag: "2d84996d8-802681-650f4df3"
Connection: close
Accept-Ranges: bytes
Content-Length: 8398465
Content-Type: application/x-apple-diskimage

Steps to Reproduce

Attempt to build/package any plugin using the current template
Watch as the build fails as soon as Packages.dmg is (not) verified.

Anything else we should know?

No response

matthewbrandt commented 11 months ago

This seems concerning to me (as an end-user).

PatTheMav commented 11 months ago

Good thing we got those hash checks - the bundle reports itself to be the same old version (so not the update that seems to have been worked on at the GitHub repo).

According to VirusTotal, the helper binaries are unchanged since last year (their hashes are identical), the privileged helper tools don't seem to have ever been checked though. None of the binaries was flagged by VirusTotal, but that doesn't need to mean much for such a recent update.

FiniteSingularity commented 11 months ago

Good thing we got those hash checks - the bundle reports itself to be the same old version (so not the update that seems to have been worked on at the GitHub repo).

According to VirusTotal, the helper binaries are unchanged since last year (their hashes are identical), the privileged helper tools don't seem to have ever been checked though. None of the binaries was flagged by VirusTotal, but that doesn't need to mean much for such a recent update.

Absolutely! Always nice when a safeguard works and catches a potential issue.

I have opened an issue over on the Packages repo, asking them to explain why the binaries changed but the version number stayed the same: https://github.com/packagesdev/packages/issues/144 . Hopefully they can provide more insight. For now, if we're trusting the original package, a temporary fix is to use the wayback link for the package URL. (https://web.archive.org/web/20230727054218/http://s.sudre.free.fr/Software/files/Packages.dmg)

FiniteSingularity commented 11 months ago

According to the issue I opened, it was a change by the developer, due to issues with how new versions of MacOS handle installers. Why this wasn't a minor version bump, I have no idea. The dev also provided a link to the old version, so we can either update the url to the old version link, or we can update the checksum. The link the dev provided goes here: http://s.sudre.free.fr/Software/files/Packages_1210_1.dmg

That said, for my plug-ins, I am planning on building an alternative MacOS packaging action, as I'm not fully comfortable with how the Packages dev is doing things. If y'all are open to PRs, I'd be happy to provide one once I have the new action working.

PatTheMav commented 11 months ago

I will actually try to implement a fully native (as in: Using Zsh and Xcode build tools only) solution this week because I echo that sentiment.

As we only need a CLI solution (and not the entire IDE) it should suffice to use a distribution template file and have CMake generate a version with the correct values read from buildspec.json.

PatTheMav commented 11 months ago

Turns out, all it takes is creating a valid Distribution specification in XML and creating the target directory structure with payloads under a prefix, followed by:

pkgbuild --identifier "<BUNDLE ID>" --version '<VERSION>' --root <OUTPUT DIRECTORY WITH TARGET DIRECTORY STRUCTURE> <TEMP_DIR>/<PRODUCT_NAME>.pkg
productbuild --distribution <DISTRIBUTION FILE GENERATED BY CMAKE> --package-path <TEMP_DIR> <RELEASE_DIR>/<PRODUCT_NAME>.pkg
productsign and notarization

The second step is important because it reads in the distribution file which contains important options:

   <options
        rootVolumeOnly="true"
        hostArchitectures="arm64,x86_64"
        customize="never"
        allow-external-scripts="no" />
    <domains enable_currentUserHome="true" enable_anywhere="false" enable_localSystem="false" />

It's also where you house custom JavaScript and the installation choices (though in this case customisation is disabled) and the domains part ensures that plugins are only installed to the user library and not the system library.

FiniteSingularity commented 11 months ago

Fantastic! So glad you are disabling the ability to install globally, and are requiring the install be in the user library. That is one of the bugs I have in my issues queue, as I've had a few users try to install the plugin globally, but then it doesn't work.

PatTheMav commented 11 months ago

The current packages template file already limits to the user domain for installations as well (so no elevated permission prompt on installation), I think I changed that a while back because I don't want to have that stuff in the system library.

Though technically OBS should still load them from both locations because we wanted to retain compatibility with some older plugins on x86_64, but I'd have to check if Apple Silicon also loaded them from there.

FiniteSingularity commented 11 months ago

Though technically OBS should still load them from both locations because we wanted to retain compatibility with some older plugins on x86_64, but I'd have to check if Apple Silicon also loaded them from there.

In my testing on a Apple Silicon Mac, unfortunately it doesn't load from the global library install location.

I think I'm using an older set of cmake files that allow for in-tree builds, as I've not been able to figure out how to get the Visual Studio debugger to work otherwise. I'd love to get that all figured out, and even write some docs for others (apologies for going off-topic).

gxalpha commented 11 months ago

but I'd have to check if Apple Silicon also loaded them from there.

In my testing on an Apple Silicon Mac, unfortunately it doesn't load from the global library install location.

@FiniteSingularity @PatTheMav only x86_64 OBS load plugins from legacy locations and package formats (global install directory and/or .so binaries). aarch64 OBS only loads .plugin bundles, and only from the user directory.

See https://github.com/obsproject/obs-studio/blob/aaca2b6e73b49dc0685eaa6e7d3933ca343395c0/UI/window-basic-main.cpp#L205-L226

PatTheMav commented 11 months ago

Yeah I faintly remember that - because we also introduced the new format for Apple Silicon, it didn't require to keep the old load location.

dspstanky commented 11 months ago

To add on to @FiniteSingularity 's initial post, when I try to use the wayback machine link as a workaround, for whatever reason the actual checksum changes each time I try to run a build/package and I get a mismatch error. Using the dev provided link also results in a mismatch checksum. If I just keep the baseURL at default, and change the checksum to the current versions checksum, then the package fails when trying to attach.

Not sure if this is a Github Actions issue, or somewhere along my plugin's development I changed something I shouldn't have,

Obviously if a change is coming soon that just rips Package out completely, this will probably be a moot issue for me.

PatTheMav commented 11 months ago

Just posted a PR that removes the dependency entirely.

obsproject / obs-plugintemplate