Unsupported Darwin Platform Causes Test Failures on macOS

GabLeRoux commented 3 weeks ago

Bug Description:

When committing changes on macOS (Darwin), certain tests fail due to Darwin not being recognized as a supported platform. Specifically, the ImageTag tests return errors, indicating that Darwin is unsupported, which blocks successful test execution on macOS systems.

How to Reproduce:

Commit code changes on a macOS machine.
Observe that the test suite runs, but fails on specific tests due to unsupported platform errors.

Expected Behavior:

Tests should either support the Darwin platform or handle unsupported platforms gracefully without causing test failures.

Failing Test Details:

FAIL  src/model/image-tag.test.ts
  ImageTag
    constructor
      ✕ can be called (12 ms)
      ✕ accepts parameters and sets the right properties (1 ms)
      ✕ accepts "2000.0.0f0" version format (1 ms)
      ✕ accepts "2011.1.11f1" version format (1 ms)
      ✕ accepts "6000.0.0f1" version format (2 ms)
      ✓ throws for incorrect versions "some version"
      ✓ throws for incorrect versions "" (1 ms)
      ✓ throws for incorrect versions 1
    toString
      ✕ returns the correct version
      ✕ returns customImage if given
      ✕ returns the specific build platform (1 ms)
      ✕ returns no specific build platform for generic targetPlatforms

Error example:

ImageTag › constructor › can be called

expect(received).not.toThrow()

Error name:    "Error"
Error message: "The Operating System of this runner, \"darwin\", is not yet supported."

Additional Details:

The tests should ideally support Darwin, or if that is not feasible, an alternative handling method should be implemented to prevent these failures on macOS systems.

Complete output of the `yarn test` command executed on macOS

``` yarn test yarn run v1.22.21 warning ../../../package.json: No license field $ jest watchman warning: Recrawled this watch 7 times, most recently because: MustScanSubDirs UserDroppedTo resolve, please review the information on https://facebook.github.io/watchman/docs/troubleshooting.html#recrawl To clear this warning, run: `watchman watch-del '/Users/gableroux/repos/gameci/unity-test-runner' ; watchman watch-project '/Users/gableroux/repos/gameci/unity-test-runner'` FAIL src/model/image-tag.test.ts ImageTag constructor ✕ can be called (11 ms) ✕ accepts parameters and sets the right properties ✕ accepts "2000.0.0f0" version format (3 ms) ✕ accepts "2011.1.11f1" version format (1 ms) ✕ accepts "6000.0.0f1" version format (1 ms) ✓ throws for incorrect versions "some version" ✓ throws for incorrect versions "" ✓ throws for incorrect versions 1 toString ✕ returns the correct version ✕ returns customImage if given ✕ returns the specific build platform ✕ returns no specific build platform for generic targetPlatforms ● ImageTag › constructor › can be called expect(received).not.toThrow() Error name: "Error" Error message: "The Operating System of this runner, \"darwin\", is not yet supported." 73 | return Platform.types.StandaloneWindows; 74 | default: > 75 | throw new Error( | ^ 76 | `The Operating System of this runner, "${platform}", is not yet supported.`, 77 | ); 78 | } at Function.getImagePlatformType (src/model/image-tag.ts:75:15) at new ImageTag (src/model/image-tag.ts:15:33) at src/model/image-tag.test.ts:19:20 at Object. (node_modules/expect/build/toThrowMatchers.js:83:11) at Object.throwingMatcher [as toThrow] (node_modules/expect/build/index.js:338:21) at Object. (src/model/image-tag.test.ts:19:68) 17 | it('can be called', () => { 18 | const { targetPlatform } = some; > 19 | expect(() => new ImageTag({ platform: targetPlatform })).not.toThrow(); | ^ 20 | }); 21 | 22 | it('accepts parameters and sets the right properties', () => { at Object. (src/model/image-tag.test.ts:19:68) ● ImageTag › constructor › accepts parameters and sets the right properties The Operating System of this runner, "darwin", is not yet supported. 60 | return 'windows'; 61 | default: > 62 | throw new Error( | ^ 63 | `The Operating System of this runner, "${platform}", is not yet supported.`, 64 | ); 65 | } at Function.getImagePlatformPrefix (src/model/image-tag.ts:62:15) at new ImageTag (src/model/image-tag.ts:33:41) at Object. (src/model/image-tag.test.ts:23:21) ● ImageTag › constructor › accepts "2000.0.0f0" version format expect(received).not.toThrow() Error name: "Error" Error message: "The Operating System of this runner, \"darwin\", is not yet supported." 60 | return 'windows'; 61 | default: > 62 | throw new Error( | ^ 63 | `The Operating System of this runner, "${platform}", is not yet supported.`, 64 | ); 65 | } at Function.getImagePlatformPrefix (src/model/image-tag.ts:62:15) at new ImageTag (src/model/image-tag.ts:33:41) at src/model/image-tag.test.ts:35:17 at Object. (node_modules/expect/build/toThrowMatchers.js:83:11) at Object.throwingMatcher [as toThrow] (node_modules/expect/build/index.js:338:21) at src/model/image-tag.test.ts:36:15 34 | expect( 35 | () => new ImageTag({ editorVersion, targetPlatform: some.targetPlatform }), > 36 | ).not.toThrow(); | ^ 37 | }, 38 | ); 39 | at src/model/image-tag.test.ts:36:15 ● ImageTag › constructor › accepts "2011.1.11f1" version format expect(received).not.toThrow() Error name: "Error" Error message: "The Operating System of this runner, \"darwin\", is not yet supported." 60 | return 'windows'; 61 | default: > 62 | throw new Error( | ^ 63 | `The Operating System of this runner, "${platform}", is not yet supported.`, 64 | ); 65 | } at Function.getImagePlatformPrefix (src/model/image-tag.ts:62:15) at new ImageTag (src/model/image-tag.ts:33:41) at src/model/image-tag.test.ts:35:17 at Object. (node_modules/expect/build/toThrowMatchers.js:83:11) at Object.throwingMatcher [as toThrow] (node_modules/expect/build/index.js:338:21) at src/model/image-tag.test.ts:36:15 34 | expect( 35 | () => new ImageTag({ editorVersion, targetPlatform: some.targetPlatform }), > 36 | ).not.toThrow(); | ^ 37 | }, 38 | ); 39 | at src/model/image-tag.test.ts:36:15 ● ImageTag › constructor › accepts "6000.0.0f1" version format expect(received).not.toThrow() Error name: "Error" Error message: "The Operating System of this runner, \"darwin\", is not yet supported." 60 | return 'windows'; 61 | default: > 62 | throw new Error( | ^ 63 | `The Operating System of this runner, "${platform}", is not yet supported.`, 64 | ); 65 | } at Function.getImagePlatformPrefix (src/model/image-tag.ts:62:15) at new ImageTag (src/model/image-tag.ts:33:41) at src/model/image-tag.test.ts:35:17 at Object. (node_modules/expect/build/toThrowMatchers.js:83:11) at Object.throwingMatcher [as toThrow] (node_modules/expect/build/index.js:338:21) at src/model/image-tag.test.ts:36:15 34 | expect( 35 | () => new ImageTag({ editorVersion, targetPlatform: some.targetPlatform }), > 36 | ).not.toThrow(); | ^ 37 | }, 38 | ); 39 | at src/model/image-tag.test.ts:36:15 ● ImageTag › toString › returns the correct version The Operating System of this runner, "darwin", is not yet supported. 60 | return 'windows'; 61 | default: > 62 | throw new Error( | ^ 63 | `The Operating System of this runner, "${platform}", is not yet supported.`, 64 | ); 65 | } at Function.getImagePlatformPrefix (src/model/image-tag.ts:62:15) at new ImageTag (src/model/image-tag.ts:33:41) at Object. (src/model/image-tag.test.ts:48:21) ● ImageTag › toString › returns customImage if given The Operating System of this runner, "darwin", is not yet supported. 60 | return 'windows'; 61 | default: > 62 | throw new Error( | ^ 63 | `The Operating System of this runner, "${platform}", is not yet supported.`, 64 | ); 65 | } at Function.getImagePlatformPrefix (src/model/image-tag.ts:62:15) at new ImageTag (src/model/image-tag.ts:33:41) at Object. (src/model/image-tag.test.ts:65:21) ● ImageTag › toString › returns the specific build platform The Operating System of this runner, "darwin", is not yet supported. 60 | return 'windows'; 61 | default: > 62 | throw new Error( | ^ 63 | `The Operating System of this runner, "${platform}", is not yet supported.`, 64 | ); 65 | } at Function.getImagePlatformPrefix (src/model/image-tag.ts:62:15) at new ImageTag (src/model/image-tag.ts:33:41) at Object. (src/model/image-tag.test.ts:77:21) ● ImageTag › toString › returns no specific build platform for generic targetPlatforms The Operating System of this runner, "darwin", is not yet supported. 60 | return 'windows'; 61 | default: > 62 | throw new Error( | ^ 63 | `The Operating System of this runner, "${platform}", is not yet supported.`, 64 | ); 65 | } at Function.getImagePlatformPrefix (src/model/image-tag.ts:62:15) at new ImageTag (src/model/image-tag.ts:33:41) at Object. (src/model/image-tag.test.ts:95:21) (node:32114) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead. (Use `node --trace-deprecation ...` to show where the warning was created) PASS src/model/results-check.test.ts ResultsCheck createCheck ✓ throws for missing input PASS src/model/index.test.ts Index ✓ exports Action (1 ms) ✓ exports Docker ✓ exports ImageTag ✓ exports Input ✓ exports Output ✓ exports ResultsCheck PASS src/model/results-parser.test.ts ResultsParser parseResults ✓ throws for missing file (1 ms) ✓ parses editmode-results.xml ✓ parses playmode-results.xml convertResults ✓ converts editmode-results.xml (4 ms) ✓ converts playmode-results.xml (2 ms) convertSuite ✓ convert single (1 ms) convertTests ✓ convert array ✓ convert single convertTestCase ✓ not failed ✓ no stack trace (1 ms) ✓ no cdata in stack trace ✓ no annotation path ✓ prepare annotation without console output ✓ prepare annotation with console output findAnnotationPoint ✓ keep working if not matching ✓ simple annotation point ✓ first entry with non-zero line number annotation point (1 ms) ✓ setup annotation point (2 ms) ✓ Debug.LogError annotation point (1 ms) PASS src/model/input.test.ts Input getFromUser ✓ does not throw ✓ returns an object (1 ms) isValidFolderName ✓ returns true for . ✓ returns true for ./ ✓ returns true for folder ✓ returns true for trailing/ ✓ returns true for .hidden (1 ms) ✓ returns true for .hidden/sub ✓ returns true for .hidden/trailing/ ✓ returns true for ./.hidden-sub ✓ returns true for hyphen-folder ✓ returns true for under_score ✓ returns false for .. (1 ms) ✓ returns false for ../ getPackageNameFromPackageJson ✓ throws error if package.json cannot be found at the given project path (5 ms) ✓ throws error if package.json contents cannot be parsed ✓ throws error if name field in package.json is not present (1 ms) ✓ throws error if name field in package.json is present but not a string ✓ throws error if name field in package.json is present but empty ✓ returns the name field in package.json if it is present as a non-empty string verifyTestsFolderIsPresent ✓ throws error if tests folder is not present (1 ms) ✓ does not throw if tests folder is present PASS src/model/output.test.ts Output setArtifactsPath ✓ does not throw setCoveragePath ✓ does not throw (1 ms) PASS src/model/action.test.ts Action ✓ returns the root folder of the action ✓ returns the action folder compatibility check ✓ throws for anything other than linux or windows (2 ms) Trying to open ./artifacts/editmode-results.xml Trying to open ./artifacts/playmode-results.xml Start analyzing results: editmode-results.xml ::debug::Convert data for test Editor.EditorModeTest.FailedTest ::debug::No console output for test case: Editor.EditorModeTest.FailedTest - /github/workspace/unity-project/Assets/Editor/EditorModeTest.cs:21 - Editor.EditorModeTest.FailedTest ::debug::Convert data for test Editor.EditorModeTest.FailedUnityTest ::debug::No console output for test case: Editor.EditorModeTest.FailedUnityTest - /github/workspace/unity-project/Assets/Editor/EditorModeTest.cs:40 - Editor.EditorModeTest.FailedUnityTest ::debug::Skip test Editor.EditorModeTest.IgnoredTest without failure data ::debug::Skip test Editor.EditorModeTest.IgnoredUnityTest without failure data ::debug::Skip test Editor.EditorModeTest.PassedTest without failure data ::debug::Skip test Editor.EditorModeTest.PassedUnityTest without failure data ::debug::[object Object],[object Object],[object Object],[object Object],[object Object],[object Object] Start analyzing results: playmode-results.xml ::debug::Convert data for test Tests.PlayModeTest.FailedTest ::debug::No console output for test case: Tests.PlayModeTest.FailedTest - /github/workspace/unity-project/Assets/Tests/PlayModeTest.cs:20 - Tests.PlayModeTest.FailedTest ::debug::Convert data for test Tests.PlayModeTest.FailedUnityTest ::debug::No console output for test case: Tests.PlayModeTest.FailedUnityTest - /github/workspace/unity-project/Assets/Tests/PlayModeTest.cs:39 - Tests.PlayModeTest.FailedUnityTest ::debug::Skip test Tests.PlayModeTest.IgnoredTest without failure data ::debug::Skip test Tests.PlayModeTest.IgnoredUnityTest without failure data ::debug::Skip test Tests.PlayModeTest.PassedTest without failure data ::debug::Skip test Tests.PlayModeTest.PassedUnityTest without failure data ::debug::Convert data for test Tests.SetupFailedTest.PassedTest ::debug::No console output for test case: Tests.SetupFailedTest.PassedTest - /github/workspace/unity-project/Assets/Tests/SetupFailedTest.cs:10 - Tests.SetupFailedTest.PassedTest ::debug::Convert data for test Tests.TearDownFailedTest.PassedTest ::debug::No console output for test case: Tests.TearDownFailedTest.PassedTest - /github/workspace/unity-project/Assets/Tests/TearDownFailedTest.cs:10 - Tests.TearDownFailedTest.PassedTest ::debug::[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object] ::debug::Skip test undefined without failure data ::debug::Skip test undefined without failure data ::debug::Skip test undefined without failure data ::debug::Skip test undefined without failure data ::debug::Skip test undefined without failure data ::debug::Skip test undefined without failure data ::debug::Skip test undefined without failure data ::debug::Convert data for test undefined ::warning::No stack trace for test case: undefined ::debug::Convert data for test undefined ::warning::No cdata in stack trace for test case: undefined ::debug::Convert data for test undefined ::warning::Not able to find annotation point for failed test! Test trace: Test CDATA ::debug::Convert data for test Test Full Name ::debug::No console output for test case: Test Full Name - /github/workspace/unity-project/Assets/Tests/SetupFailedTest.cs:10 - Test Full Name ::debug::Convert data for test Test Full Name - /github/workspace/unity-project/Assets/Tests/SetupFailedTest.cs:10 - Test Full Name ::set-output name=artifactsPath:: ::set-output name=coveragePath:: ::set-output name=coveragePath::artifacts ::set-output name=coveragePath::coverage ::set-output name=coveragePath::CodeCoverage ::set-output name=coveragePath::./artifacts ::set-output name=coveragePath::./coverage ::set-output name=coveragePath::./CodeCoverage ::set-output name=coveragePath::./artifacts/coverage ::set-output name=coveragePath::./coverage/ ::set-output name=coveragePath::./CodeCoverage/ ::set-output name=coveragePath::./artifacts/coverage/ File ./artifacts/playmode-results.xml parsed... Start analyzing results: playmode-results.xml ::debug::Convert data for test Tests.PlayModeTest.FailedTest ::debug::No console output for test case: Tests.PlayModeTest.FailedTest - /github/workspace/unity-project/Assets/Tests/PlayModeTest.cs:20 - Tests.PlayModeTest.FailedTest ::debug::Convert data for test Tests.PlayModeTest.FailedUnityTest ::debug::No console output for test case: Tests.PlayModeTest.FailedUnityTest - /github/workspace/unity-project/Assets/Tests/PlayModeTest.cs:39 - Tests.PlayModeTest.FailedUnityTest ::debug::Skip test Tests.PlayModeTest.IgnoredTest without failure data ::debug::Skip test Tests.PlayModeTest.IgnoredUnityTest without failure data ::debug::Skip test Tests.PlayModeTest.PassedTest without failure data ::debug::Skip test Tests.PlayModeTest.PassedUnityTest without failure data ::debug::Convert data for test Tests.SetupFailedTest.PassedTest ::debug::No console output for test case: Tests.SetupFailedTest.PassedTest - /github/workspace/unity-project/Assets/Tests/SetupFailedTest.cs:10 - Tests.SetupFailedTest.PassedTest ::debug::Convert data for test Tests.TearDownFailedTest.PassedTest ::debug::No console output for test case: Tests.TearDownFailedTest.PassedTest - /github/workspace/unity-project/Assets/Tests/TearDownFailedTest.cs:10 - Tests.TearDownFailedTest.PassedTest ::debug::[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object] File ./artifacts/editmode-results.xml parsed... Start analyzing results: editmode-results.xml ::debug::Convert data for test Editor.EditorModeTest.FailedTest ::debug::No console output for test case: Editor.EditorModeTest.FailedTest - /github/workspace/unity-project/Assets/Editor/EditorModeTest.cs:21 - Editor.EditorModeTest.FailedTest ::debug::Convert data for test Editor.EditorModeTest.FailedUnityTest ::debug::No console output for test case: Editor.EditorModeTest.FailedUnityTest - /github/workspace/unity-project/Assets/Editor/EditorModeTest.cs:40 - Editor.EditorModeTest.FailedUnityTest ::debug::Skip test Editor.EditorModeTest.IgnoredTest without failure data ::debug::Skip test Editor.EditorModeTest.IgnoredUnityTest without failure data ::debug::Skip test Editor.EditorModeTest.PassedTest without failure data ::debug::Skip test Editor.EditorModeTest.PassedUnityTest without failure data ::debug::[object Object],[object Object],[object Object],[object Object],[object Object],[object Object] PASS src/model/unity-version-parser.test.ts UnityVersionParser parse ✓ throws for empty string (1 ms) ✓ parses from ProjectVersion.txt ✓ parses Unity 6000 and newer from ProjectVersion.txt read ✓ throws for invalid path ✓ reads from unity-project-with-correct-tests PASS src/model/platform.test.ts Platform default ✓ does not throw ✓ returns a string ✓ returns a platform isWindows ✓ returns true for windows ✓ returns false for MacOS isAndroid ✓ returns true for Android (1 ms) ✓ returns false for Windows Test Suites: 1 failed, 1 skipped, 8 passed, 9 of 10 total Tests: 9 failed, 1 skipped, 68 passed, 78 total Snapshots: 0 total Time: 1.218 s Ran all test suites. error Command failed with exit code 1. info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command. ```

This works fine on Windows or Linux, but we cannot commit easily on macOS at this time because husky runs the tests when we commit. Workaround is to use --no-verify parameter when we commit on this project, which is not ideal.

Example:

# Fix some code
git add .
git commit -m "Fix tests" --no-verify
# build the project by hand
yarn build
git add dist
git commit -m "yarn build" --no-verify

GabLeRoux commented 3 weeks ago

@coderabbitai based on these observations, what do you recommend? Considering the CI runs on Linux and will catch failing tests, should we skip the tests when we detect we're on Darwin? If so, please suggest a fix.

coderabbitai[bot] commented 3 weeks ago

[!TIP] For best results, initiate chat on the files or code changes.

@GabLeRoux, based on your observations, I recommend adjusting the tests to ensure they pass on macOS without skipping them. Since the CI runs on Linux and will catch any platform-specific issues, making the tests platform-independent where possible will improve the developer experience on macOS.

One effective approach is to mock the process.platform value during the tests so that it returns a supported platform like 'linux'. This way, when running tests on macOS, the platform-dependent code won't throw errors due to the unsupported 'darwin' platform.

Here's how you can modify your image-tag.test.ts file:

// At the top of your image-tag.test.ts file
const originalPlatform = process.platform;

beforeAll(() => {
  Object.defineProperty(process, 'platform', {
    value: 'linux',
  });
});

afterAll(() => {
  Object.defineProperty(process, 'platform', {
    value: originalPlatform,
  });
});

This code temporarily overrides process.platform to 'linux' before the tests run and restores the original value afterward. This allows the tests to execute successfully on macOS without affecting the actual platform detection in your production code.

Alternatively, if you prefer to conditionally skip the platform-specific tests on unsupported platforms, you can adjust your test suites as follows:

const isSupportedPlatform = ['linux', 'win32'].includes(process.platform);

if (isSupportedPlatform) {
  describe('ImageTag', () => {
    // Place your tests here
  });
} else {
  describe.skip('ImageTag', () => {
    // Tests are skipped on unsupported platforms
  });
}

However, mocking process.platform is generally preferable because it ensures that your tests are run and validated consistently across different development environments.

Let me know if this solution works for you or if you need further assistance!

🔗 Analysis chain

🏁 Scripts executed

The following scripts were executed for the analysis: Script: ```shell #!/bin/bash # Find and output the contents of 'image-tag.ts' and 'platform.ts' to analyze platform detection logic fd 'image-tag.ts' --type f --exec cat {} fd 'platform.ts' --type f --exec cat {} ``` Length of output: 5643

game-ci / unity-test-runner

Unsupported Darwin Platform Causes Test Failures on macOS #287