Make snapshot testing work for testing sandboxed macOS apps

Hi guys,

First of all, thanks for the great project (and the cool videos)! I'm trying to get snapshot testing to work with unit tests for a sandboxed macOS Cocoa application.

I'm running into a problem:

The unit tests are for testing my application target (not a framework) and hence run hosted inside my sandboxed app. Because of that, the snapshots directory in the project folder is not accessible to my unit tests and recording a snapshot fails with: Code=513 "You don’t have permission to save the file “TestOutput” in the folder “Snapshots”."

This problem does not exist when running unit tests for a framework target. Those are not sandboxed.

I considered a few things, but maybe you have a better idea:

Option 1: Move code to be tested into a framework target While technically possible, not always practical…

Option 2: Don't run unit tests inside the hosting application Tests don't run sandboxed this way, but we lose access to testing internal types and methods (@testable import requires unit tests to run inside a hosting application and the "Allow testing Host Application APIs" flag to be set.

Option 3: Copy snapshots to temporary directory before tests start, copy back once done The idea:

Since sandboxed unit tests are allowed to read and write to the system temporary directory, copy the snapshots into a well-known location inside the temporary directory before the tests run.
Once the tests finish, copy the newly generated snapshots back from the temp directory into the project source directory.

Xcode allows to run run scripts pre and post test execution. I'm attaching a shell script that copies the snapshots around as a starting point. The script contains more details how to set this up.

tests-pre-post-action.sh.txt

To get this to work, the snapshot code needs one adjustment:

https://github.com/pointfreeco/swift-snapshot-testing/blob/fb70100e9f15e014bd0707575756f6812e2ec827/Sources/SnapshotTesting/AssertSnapshot.swift#L174

let envSnapshotsDir = ProcessInfo.processInfo.environment["USE_TEMP_SNAPSHOTS_DIR"] != nil
                  ? URL(fileURLWithPath: NSTemporaryDirectory(), isDirectory: true)
                    .appendingPathComponent("__Snapshots__")
                    .appendingPathComponent(fileName)
                  : nil

let snapshotDirectoryUrl = snapshotDirectory.map { URL(fileURLWithPath: $0, isDirectory: true) }
?? envSnapshotsDir
?? fileUrl
  .deletingLastPathComponent()
  .appendingPathComponent("__Snapshots__")
  .appendingPathComponent(fileName)

Option 4? Extend sandbox somehow It would be great if we could simply pass the path to the snapshots directory as an argument to the executable running the unit tests. I'm not aware that extending the app sandbox this way is possible, but maybe something along those lines?

pointfreeco / swift-snapshot-testing

Make snapshot testing work for testing sandboxed macOS apps #277