Several Questions Related to Bluepill

[Husseinovich]

I have a few questions:

1. I have the options - "screenshots-directory": "./screen/". However the screen's folder are empty despite some of the tests failing. Is there anything else that i need to do?
2. What is the latest most updated fastlane plugin for Bluepill that you guys are using?

Below is my config file:

  {
 "xctestrun-path": "path to xctestrun", 
   "output-dir": "/Users/Testresults",
   "device": "iPhone 11 Pro",
   "num-sims": 2,
   "screenshots-directory": "./screen/",
   "reuse-simulator": true,
   "failure-tolerance": 3,
   "only-retry-failed": true
}

[ob]

For the screenshots not being created, can you paste the log from bluepill? You might have to run it in verbose mode to get more information. Your output dir looks a little weird... do you have permission to write in /Users/Testresults?

We don't use fastlane, but I'm pretty sure there is a plugin for bluepill created by the community.

[Husseinovich]

@ob Thanks for the follow up. Here is the updated config.json

{
   "xctestrun-path": "bluepill/Build/Products/collate_errors_iphonesimulator13.2-x86_64.xctestrun", 
   "device": "iPhone 11 Pro",
   "num-sims": 2,
   "reuse-simulator": true,
   "failure-tolerance": 1,
   "only-retry-failed": true,
   "output-dir": "test_results/",
   "screenshots-directory": "screen/"
}

The screen folder is created but its empty. My thinking is that failed test cases would be saved there. test_results folder is within the project folder and is successfully created with the html report and other artifacts. Does the html report attach failed test cases screenshots? Attached verbose logs logs.txt

[Husseinovich]

@ob Did you get a chance to look at it? Thanks

[ob]

Sorry, I got distracted ;-)

So from your logs:

[20:44:28]: ▸ {12302} 20200318.204428 [ DEBUG  ] (BP-1) [CONFIGURATION] screenshotsDirectory: /Users/baxio/Downloads/collate_errors-master/fastlane/screen

I assume there are no screenshots in that directory?

[Husseinovich]

@ob Correct there are no screenshots in the directory but the directory itself is created successfully.

[ob]

Ok, can you try something? Can you try changing this line:

https://github.com/linkedin/bluepill/blob/42a4b8c9b8d07f58d55872aa45360730a858a768/bp/src/SimulatorMonitor.m#L250

from a SIGKILL to a SIGTERM and see if that fixes it?

[Husseinovich]

@ob Can I installed via homebrew, do you want me to install the binaries to be able to make this change?

[ob]

Ah, sorry yeah... you'd have to build from source to try it.

[Husseinovich]

@ob I made the change and build from sources and still screenshots folder is empty.

[Husseinovich]

@ob Do you know if there is a way to specify more than one type of device, for example"

devices: ["iPhone 11", "iPad Pro"]

If not, then what is the best way to execute same tests for both iPhone and Tablet without having to them sequentially?

[Husseinovich]

@ob Sorry, just wanted to check if you got a chance to respond :)

[ob]

Sorry, I've been swamped with something else ;-)

Bluepill doesn't currently support multiple devices. Can you file an issue asking for this feature?

[chenxiao0228]

@Husseinovich thanks reporting this issue.

Can you please open an feature request?