MobileNativeFoundation / bluepill

Bluepill is a reliable iOS testing tool that runs UI tests using multiple simulators on a single machine
BSD 2-Clause "Simplified" License
3.18k stars 231 forks source link

(BLUEPILL) Failed to read data for UIABCTests - ERROR: Failed to load some test bundles #544

Closed parikhparth23 closed 1 year ago

parikhparth23 commented 1 year ago

Hello, First time using bluepill.

I installed bluepill through brew install bluepill

I build my app by running the following command: xcodebuild -workspace MY.xcworkspace -scheme "UIABCTests" -sdk iphonesimulator -destination 'platform=iOS Simulator,name=iPhone 14 Pro Max' build-for-testing | xcbeautify

I got the xctestrun from Library/Developer/Xcode/DerivedData/name/Build/Products/UIABCTests_iphonesimulator16.0-arm64.xctestrun

AND the app from Library/Developer/Xcode/DerivedData/name/Build/Products/Debug-iphonesimulator/NameApp.app

Running this command:

     bluepill --xctestrun-path ./UIABCTests_iphonesimulator16.0-arm64.xctestrun -o ./output/

OR

{ 
   "app": "~/Desktop/NameApp.app",
   "xctestrun-path": "~/Desktop/UIABCTests_iphonesimulator16.0-arm64.xctestrun",
   "output-dir": "./build/" 
}

Also tried adding "unsafe-skip-xcode-version-check": "yes" to the abovejson` but did not work. and also tried

bluepill --unsafe-skip-xcode-version-check YES --xctestrun-path UIABCTests_iphonesimulator16.0-arm64.xctestrun -o ./output

no luck either :(.

gives me this error

2022-10-28 18:35:16.764 bluepill[37085:345815] Debug Enabled == 0
{37085} 20221028.183517 [  INFO  ] (BLUEPILL) Using xctestrun configuration
{37085} 20221028.183517 [  ERROR ] (BLUEPILL) Failed to read data for UIABCTests
ERROR: Failed to load some test bundles

bluepill version: Bluepill v5.12.1 xcode version: Version 14.0.1 (14A400) OS: 13.0 (22A380)

parikhparth23 commented 1 year ago

@oliverhu can you plz help with this issue?

ob commented 1 year ago

Can you enable verbose logging to see why it might have failed? It's the --verbose flag. Thanks!

ob commented 1 year ago

Also, we should print errPtr here that would help debug this.

parikhparth23 commented 1 year ago

@ob running with --verbose flag

bluepill -c config.json --verbose
2022-10-31 10:26:13.790 bluepill[2437:23880] Debug Enabled == 1
{2437} 20221031.102614 [ DEBUG  ] (BLUEPILL) xcode build version: Xcode 14.0.1
Build version 14A400
{2437} 20221031.102614 [  INFO  ] (BLUEPILL) Using xctestrun configuration
{2437} 20221031.102614 [  ERROR ] (BLUEPILL) Failed to read data for UIEventsTests
ERROR: Failed to load some test bundles
parikhparth23 commented 1 year ago

@ob i am running the brew version and not compiling locally. So I don't think adding errPtr will help.

ob commented 1 year ago

Could you post your .xctestrun file?

parikhparth23 commented 1 year ago

@ob sure, let me upload it here

parikhparth23 commented 1 year ago

Hi Oscar, Here is the App, config.JSON and xctestrun app, xctestrun, config.json.zip

Full app - Simple App.zip

command i used:

xcodebuild -workspace simpleapp.xcworkspace -scheme "Simple App" -sdk iphonesimulator -destination 'platform=iOS Simulator,name=iPhone 14 Pro Max' build-for-testing | xcbeautify
 bluepill -c config.json --verbose

i am getting the same error: bluepill -c config.json --verbose

Debug Enabled == 1
{6569} 20221031.114948 [ DEBUG  ] (BLUEPILL) xcode build version: Xcode 14.0.1
Build version 14A400
{6569} 20221031.114948 [  INFO  ] (BLUEPILL) Using xctestrun configuration
{6569} 20221031.114948 [  ERROR ] (BLUEPILL) Failed to read data for Simple AppTests
{6569} 20221031.114948 [  ERROR ] (BLUEPILL) Failed to read data for Simple AppUITests
ERROR: Failed to load some test bundles

bluepill version: Bluepill v5.12.1 xcode version: Version 14.0.1 (14A400) OS: 13.0 (22A380)

parikhparth23 commented 1 year ago

HI @ob , would appreciate your response.

ob commented 1 year ago

Just eyeballing the .xctestrun file it looks correct... without printing out the errPtr where I mentioned in this comment it'll be very hard to figure out what's going on... there's too many code paths.

I would suggest either compiling bluepill from source and adding the missing errPtr (PR welcome) or waiting for me to add it and producing a new release.

ob commented 1 year ago

So I just tried it with the app you uploaded and it seems to work. All I did was download Simple App and run:

 xcodebuild -workspace simpleapp.xcworkspace -scheme "Simple App" -sdk iphonesimulator -destination 'platform=iOS Simulator,name=iPhone 14 Pro Max' build-for-testing

then

bluepill --xctestrun-path '/Users/ob/Library/Developer/Xcode/DerivedData/simpleapp-fzrsvuwogsmknrhjifszqxxscbin/Build/Products/Simple App_iphonesimulator16.0-arm64.xctestrun' --output-dir ./build

and it works for me. This is with the latest bluepill (built from master) and Xcode 14.0.1. I even diffed the .xctestrun file produced by Xcode for my build with the one you uploaded and they both look identical.

Like I said before, I think you'll need to build it and fix the errPtr to see what's going on in your machine.

parikhparth23 commented 1 year ago

@ob , thank you for trying on your end. let me try on my end

parikhparth23 commented 1 year ago

@ob after building the latest(master) locally, the test ran fine....thank you for the help.

I am building my app using the following command: bazelisk build --config=Debug /MyApp

how do I run bluepill with bazel?

bluepill --test-plan-path '<WHAT IS THE VALUE>' --output-dir ./build

What is the value I need to use for test-plan-path?

I am not able to understand this from the README:

test-plan-path ->

The path of a json file which describes the test plan. 
It is equivalent to the .xctestrun file generated by Xcode, 
but it can be generated by a different build system, e.g. Bazel
chenxiao0228 commented 1 year ago

@ob after building the latest(master) locally, the test ran fine....thank you for the help.

I am building my app using the following command: bazelisk build --config=Debug /MyApp

how do I run bluepill with bazel?

bluepill --test-plan-path '<WHAT IS THE VALUE>' --output-dir ./build

What is the value I need to use for test-plan-path?

I am not able to understand this from the README:

test-plan-path ->

The path of a json file which describes the test plan. 
It is equivalent to the .xctestrun file generated by Xcode, 
but it can be generated by a different build system, e.g. Bazel

@parikhparth23 good question, we should have put some documentation for the Bazel rule.

There are three steps

  1. you need to import the Bluepill package in your WORKSPACE as a repository rule. e.g

    http_archive(
    name = "bptestrunner",
    sha256 = "6b11eb3e36680efa9f183daf1d932477c696e998d0cdf3ba3798bf8b98c05531",
    url = "https://github.com/MobileNativeFoundation/bluepill/releases/download/v5.11.0/Bluepill-v5.11.0.zip",
    strip_prefix = "Bluepill-v5.11.0",
    )
  2. load the bptestrunner rule in your BUILD file

    load("@bptestrunner//:bluepill_batch_test.bzl", "bluepill_batch_test")
  3. have a bluepill test rule in your BUILD file

    bluepill_batch_test(
    name = "FooTests",
    test_targets = [
        "BarTests" # an ios_unit_test or ios_ui_test rule from rules_apple or rules_ios
    ],
    config_file = "baz/config.json",  # path to the bluepill config file
    )
parikhparth23 commented 1 year ago

Thank you @chenxiao0228

let me try those steps

chenxiao0228 commented 1 year ago

@parikhparth23 you'd need to change the arguments of the bptestrunner repository rule and use a newer version of Bluepil though.

parikhparth23 commented 1 year ago

thank you @chenxiao0228, will change the arguments of the bptestrunner repository rule and use a newer version of Bluepill

parikhparth23 commented 1 year ago

@chenxiao0228 what if i want to use locally build version?

parikhparth23 commented 1 year ago

Above solution mentioned by @ob worked. closing this issue.

chenxiao0228 commented 1 year ago

@chenxiao0228 what if i want to use locally build version?

You can run the same commands in Bluepill's repo that builds the release archive, then you can either