search

stanfy / spoon-gradle-plugin

Gradle plugin for Spoon.
Apache License 2.0
353 stars 79 forks source link

Test process won't finish - spoon is waiting to grab screenshots #134

Open FisherKK opened 7 years ago

FisherKK commented 7 years ago

I have faced this problem many times - one year ago too. It only occurred when I used more than one device and didn't use screenshot option. It happened either on real devices or emulators. To fix it I had to be sure that folder from which spoon wants to grab screenshots exists, otherwise it was waiting until screenshot grab will succeed. So my workaround for this was creating empty folder for screenshots and this solved my problem.

Now I am facing it again and I am not sure about the cause. I can start my test session on 4 AVDs launched like that:

  1. Before each test session I delete and create new AVD devices:
android create avd --name "Test-Emulator-API23-Nexus-5" --target "Google Inc.:Google APIs:23" --abi google_apis/x86_64 --device "Nexus 5" --skin "1080x1920" --sdcard 128M
android create avd --name "Test-Emulator-API23-Nexus-5-1" --target "Google Inc.:Google APIs:23" --abi google_apis/x86_64 --device "Nexus 5" --skin "1080x1920" --sdcard 128M
android create avd --name "Test-Emulator-API23-Nexus-5-2" --target "Google Inc.:Google APIs:23" --abi google_apis/x86_64 --device "Nexus 5" --skin "1080x1920" --sdcard 128M
android create avd --name "Test-Emulator-API23-Nexus-5-3" --target "Google Inc.:Google APIs:23" --abi google_apis/x86_64 --device "Nexus 5" --skin "1080x1920" --sdcard 128M
  1. And launch them:
emulator -port 5556 -avd Test-Emulator-API23-Nexus-5 -gpu on &>/dev/null &
emulator -port 5560 -avd Test-Emulator-API23-Nexus-5-1 -gpu on &>/dev/null &
emulator -port 5562 -avd Test-Emulator-API23-Nexus-5-2 -gpu on &>/dev/null &
emulator -port 5564 -avd Test-Emulator-API23-Nexus-5-3 -gpu on &>/dev/null &

When all test finish, spoon wants to grab screenshots from logs (but I am not doing any screenshots). And SOMETIMES (like once per 6 test sessions) it freezes and there is a full log:

https://gist.github.com/FisherKK/edf9048a73aba84450a4ee9ca73e1944

Nothing happens on AVDs at that time. They are fully functional, you can interract them yet as you can see it is still waiting:

[09:50:49]: ▸ 2016-11-17 09:50:49 [SR$1.run] [emulator-5558] Execution done. (1 remaining [emulator-5556])

And that one remains forever.

I would ask for:

FisherKK commented 7 years ago

I've got update in that issue.

As I said I re-create all 4 AVDs before each test session. When test runs on AVD it checks if folder for screenshot exists - otherwise creates empty folder. But I am also using sharding and I have 3 tests in package but 4 AVDs. Because of that one AVD is not launching anything and doesn't have it's screenshot folder check. It doesn't exist so spoon always freezes on it during screenshot pull process.

I can fix it on my side but I guess that's an issue anyway?

roman-mazur commented 7 years ago

Pull request is welcome. Unfortunately, I cannot help you with code atm.

artem-zinnatullin commented 7 years ago

We've faced same issue, will work on PR on next week then.

FisherKK commented 7 years ago

Hello @artem-zinnatullin ! Did you manage to solve this issue maybe? :)

artem-zinnatullin commented 7 years ago

"In progress" in our Jira for 2 days now…

Current findings:

mirceanis commented 7 years ago

Hello all,

Just in case it helps anyone, I'll share a warstory. I was causing all the tests on a particular device to be filtered out and spoon kept hanging.

I don't remember exactly if I was filtering the tests using arguments or by causing AssumptionViolatedException but maybe one of these scenarios gives you ideas for debugging.

FisherKK commented 7 years ago

Hello,

Any does anyone has any progress in that issue?

artem-zinnatullin commented 7 years ago

So, what I've found so far is that enabling test sharding (which is what we want) fixes this problem, problem was reproducible only if Spoon run same tests across all connected emulators.

FisherKK commented 7 years ago

I have it opposite:

If I run same test set on each device then there is no problem. If I use shards it sometimes freezes - like in the Log I have sent. One AVD didn't receive any test as shards were divided that way and it froze.

I used Adb time out to 10 mins, so if something freezes I won't need to restart session. But lately I am thinking more and more about my own test launcher from scratch.

artem-zinnatullin commented 7 years ago

Damn it, just saw two CI with multiple emulators each (on different machines, in Docker, 100% isolated) jobs stuck:

2017-02-06 13:39:09 [SDR.printStream] [emulator-5554] STDOUT 2017-02-06 13:39:09 [SDR.handleImages] Moving screenshots to the image folder on [emulator-5554]

This is driving me nuts…

artem-zinnatullin commented 7 years ago

But lately I am thinking more and more about my own test launcher from scratch.

I know that feel bro… Also there are some design problems in Spoon which I would like to fix but they'll require almost complete rewrite: