Failed test in GitHub Actions when local test run on my Mac works well

[bigMOTOR]

Describe the bug I have failed test in GitHub Actions and have no idea where to dig because the local test runs on my Mac well with success result. I use Combine on lates MacOS, maybe this is the reason. But I consider it as a bug because I believe local and CI’s MacOS behavior should be the same.

Error:2020-03-06 08:59:53.328 xcodebuild[1366:11971] [MT] IDEResultKit: Warning: While writing a result bundle to /Users/runner/Library/Developer/Xcode/DerivedData/AirConnect-aokmlrgaakifmbgovglpmzlxlfhq/Logs/Test/Run-AirConnect-2020.03.06_08-59-13-+0000.xcresult, not all contents have been imported, as the Staging directory still contains the following files/directories: ["2_Test"]630Testing failed:631632Test session results, code coverage, and logs:633/Users/runner/Library/Developer/Xcode/DerivedData/AirConnect-aokmlrgaakifmbgovglpmzlxlfhq/Logs/Test/Run-AirConnect-2020.03.06_08-59-13-+0000.xcresult634635AirConnectTests:636AirConnect.app (1668) encountered an error (Early unexpected exit, operation never finished bootstrapping - no restart will be attempted. (Underlying error: Crash: AirConnect (1668): Namespace SIGNAL, Code 0x4))637638** TEST FAILED **639640Testing started on 'My Mac'641##[error]Process completed with exit code 65.

Area for Triage:

contains: 'Apple' addLabels: ['Area: Apple'] assign: ['sergey-akhalkov']

Question, Bug, or Feature?: Bug

Virtual environments affected

• [+] macOS 10.15
• [ ] Ubuntu 16.04 LTS
• [ ] Ubuntu 18.04 LTS
• [ ] Windows Server 2016 R2
• [ ] Windows Server 2019

[al-cheb]

Hello, @bigMOTOR Could you please provide steps to reproduce the issue?

[bigMOTOR]

Hi @al-cheb, It's quite difficult to provide STR because they are just "run the action on this particular repo", but this repo is private. I tried to copy some pieces of this code, but the issue didn't reproduce there. So I suggest that the simplest solution will be - just invite a person who will be assigned for this task into my repo. Does it work for you?

[bigMOTOR]

@al-cheb btw Could you help me, please. Is it correct a workflow to run the tests under MacOS on GitHub env:

name: Swift MacOS - Build and Run tests

on: [push]

jobs:
  test:
    name: Test
    runs-on: macos-latest

    steps:
      - uses: actions/checkout@v2
      - name: Build and test
        run: |
          xcodebuild clean test -project AirConnect.xcodeproj -scheme AirConnect -destination 'platform=OS X,arch=x86_64' CODE_SIGN_IDENTITY="" CODE_SIGNING_REQUIRED=NO ONLY_ACTIVE_ARCH=NO

[al-cheb]

@bigMOTOR Could you please provide logs with --verbose parameter?

[bigMOTOR]

@al-cheb please take a look: AirConnect Actions Log

[al-cheb]

@bigMOTOR Try to add env variable at a buildind step:

 env:
       FASTLANE_EXPLICIT_OPEN_SIMULATOR : "2"

[bigMOTOR]

@al-cheb Thanks for your help! I tried. Unfortunately it doesn't help.

name: Swift MacOS - Build and Run tests

on: [push]

jobs:
  test:
    name: Test
    runs-on: macos-latest

    steps:
      - uses: actions/checkout@v2
      - name: Build and test
        env: 
          FASTLANE_EXPLICIT_OPEN_SIMULATOR: "2"
        run: |
          xcodebuild clean test -project AirConnect.xcodeproj -scheme AirConnect -destination 'platform=OS X,arch=x86_64' CODE_SIGN_IDENTITY="" CODE_SIGNING_REQUIRED=NO ONLY_ACTIVE_ARCH=NO -verbose

[dmitry-shibanov]

Hello, @bigMOTOR, What version of Xcode do you use when you built locally. Also please pay attention that software on your local machine can be different from our VM images. Please find environments of macOS images on link. Based on the logs, it looks like that your tests could be stacked with some modal ui window. Can it be related to your tests? Also it would be great if you can provide small project with repro steps that hang.

[bigMOTOR]

Hi @dmitry-shibanov , You were absolutely right! I added 'Testing' buildConfiguration and disabled any work for it. Now all unit tests passed fine. Thank you for your help. I didn't think that some blocking modal window on the app target will fail unit testing.

[al-cheb]

@bigMOTOR Could we close the issue?

[bigMOTOR]

@al-cheb Sure. @al-cheb @dmitry-shibanov thanks a lot for your help!

[umair2020]

@bigMOTOR @al-cheb @dmitry-shibanov I am experiencing similar issue. The tests are runnig fine when I run on docker from my MAC but on gitHub actions some tests are failing. Looks like the wait functions are not working on gitAction but I am not sure

[renaudjenny]

Folks, I searched for a very long time to find a working configuration with the Xcode target signing configuration and the good command to execute build and test for macOS target.

Here is something working for me: On the app target (not the test target), select Sign to Run Locally for the Signing Certificate

On the test target (not the app target), select Development for the Signing Certificate

This one is tricky, for some reason, you need to set something on the iOS test target. Set Sign to Run Locally for the Signing Certificate.

And finally, here is the command to execute in the *.yml file

run: xcodebuild clean test -scheme "MemoArt (macOS)" CODE_SIGNING_ALLOWED=NO

Important point is to add the CODE_SIGNING_ALLOWED=NO

Hopefully that will work for you too 🤞.

Here is the project repo: https://github.com/renaudjenny/MemoArt

I'm able to run both iOS and macOS build and test for the same project.

[jasma-balasangameshwara]

@bigMOTOR , Please elaborate on 'You were absolutely right! I added 'Testing' buildConfiguration and disabled any work for it.". The GitActions Pytests are failing, however, they work fine on the local machine.

[deutschktw]

In my case, a unit test failed on CI runner and that caused the similar error. The failed unit test needs to connect to our server. I disabled the unit test and could solve the issue.

[OmarJalil]

I'm getting the exact same error @bigMOTOR. Did you manage to find a solution?

[macdrevx]

I'm seeing a very similar error involving 2_Test as in the original post. Examples:

• https://github.com/GEOSwift/GEOSwift/actions/runs/4690169749/jobs/8324204793
• https://github.com/GEOSwift/GEOSwift/actions/runs/4704835330/jobs/8344803947
• https://github.com/GEOSwift/GEOSwift/actions/runs/4704836061/jobs/8344805174

Failures started for me with runner image macos-11 version 20230411.1.

macos-11 version 20230328.2, was not having this problem.