search

eclipse-platform / eclipse.platform.swt

Eclipse SWT
https://www.eclipse.org/swt/
Eclipse Public License 2.0
118 stars 136 forks source link

ScreenshotOnFailure doesn't work #1518

Open jukzi opened 3 weeks ago

jukzi commented 3 weeks ago

the ScreenshotOnFailure TestWatcher produces only black images on failed tests because the @After org.eclipse.swt.tests.junit.Test_org_eclipse_swt_widgets_Widget.tearDown() calls shell.dispose(); before Screenshot is taken

reproducable with the failing Test_org_eclipse_swt_custom_StyledText

jukzi commented 3 weeks ago

see https://stackoverflow.com/questions/18008241/run-a-testwatcher-before-the-after https://github.com/junit-team/junit4/issues/906 https://www.thinkcode.se/blog/2012/07/08/performing-an-action-when-a-test-fails

basilevs commented 3 weeks ago

@jukzi is this reproducible locally? Does it happen on Windows only?

jukzi commented 3 weeks ago

if i force a test to fail, i can locally reproduce that only the desktop is captured but not the failed junit test (expected to show the wrong color). One the one hand that shows that the ScreenshotOnFailure does not work as intended. On the other hand the dark screen captured on the i build may indicate some screensaver on the server.

tomaswolf commented 3 weeks ago

@jukzi : we ran into this screenshot taking issue (screenshot taken after the @after have run) in the EGit UI tests back in 2016. Our solution back then was to amend the SwtBotJUnit4ClassRunner to take the screenshot before the @afters. Maybe the things we did in that commit might be useful here, too.

jukzi commented 3 weeks ago

if some @After/https://github.com/before method fails or a rule such as @ExpectedException fails the test. In those cases, the behavior is unchanged: these screenshots are still taken after @after has run.

@tomaswolf thanks, i like that idea.

jukzi commented 2 weeks ago

SWTBotJunit4ClassRunner looks a bit like TracingSuite already used in AllBrowserTests. Maybe that TracingSuite could be enhanced to take screenshots also on fail and be used in other suits as well.

iloveeclipse commented 2 weeks ago

@jukzi : some of the recent changes in this area broke a lot of UI & SWT tests.

Typical stack from https://download.eclipse.org/eclipse/downloads/drops4/I20241013-1800/testresults/html/org.eclipse.swt.tests_ep434I-unit-cen64-gtk3-java17_linux.gtk.x86_64_17.html

Cannot invoke "java.util.function.Supplier.get()" because "this.shell" is null

java.lang.NullPointerException: Cannot invoke "java.util.function.Supplier.get()" because "this.shell" is null
at org.eclipse.test.Screenshots$ScreenshotOnFailure.dispose(Screenshots.java:52)
at org.eclipse.test.Screenshots$ScreenshotOnFailure.finished(Screenshots.java:47)

Could you please fix it?

jukzi commented 2 weeks ago

Can u please just add a obvious null check. (I am afk)

Jörg Kubitz

Am 14.10.2024 um 10:03 schrieb Andrey Loskutov @.***>:

 @jukzi : some of the recent changes in this area broke a lot of UI & SWT tests.

Typical stack from https://download.eclipse.org/eclipse/downloads/drops4/I20241013-1800/testresults/html/org.eclipse.swt.tests_ep434I-unit-cen64-gtk3-java17_linux.gtk.x86_64_17.html

Cannot invoke "java.util.function.Supplier.get()" because "this.shell" is null

java.lang.NullPointerException: Cannot invoke "java.util.function.Supplier.get()" because "this.shell" is null at org.eclipse.test.Screenshots$ScreenshotOnFailure.dispose(Screenshots.java:52) at org.eclipse.test.Screenshots$ScreenshotOnFailure.finished(Screenshots.java:47) Could you please fix it?

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

iloveeclipse commented 2 weeks ago

Can u please just add a obvious null check. (I am afk)

OK, see https://github.com/eclipse-platform/eclipse.platform.releng.aggregator/pull/2452

jukzi commented 2 weeks ago

Thanks!

Jörg Kubitz

Am 14.10.2024 um 10:38 schrieb Andrey Loskutov @.***>:

 Can u please just add a obvious null check. (I am afk)

OK, see eclipse-platform/eclipse.platform.releng.aggregator#2452

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

jukzi commented 3 days ago

The screenshots are still not written. error is: "no screen devices"

Exception in thread "main" java.awt.AWTError: no screen devices
    at java.desktop/sun.awt.Win32GraphicsEnvironment.getDefaultScreenDevice(Win32GraphicsEnvironment.java:101)
    at java.desktop/java.awt.Robot.<init>(Robot.java:98)
    at org.eclipse.test.AwtScreenshot.main(AwtScreenshot.java:41)

https://ci.eclipse.org/releng/job/AutomatedTests/job/ep434I-unit-win32-java17/84/execution/node/3/ws/workarea/I20241026-1800/eclipse-testing/results/ep434I-unit-win32-java17_win32.win32.x86_64_17/org.eclipse.swt.tests.junit.AllNonBrowserTests.txt

jukzi commented 2 days ago

"no screen devices" may relate to https://bugs.openjdk.org/browse/JDK-8336862

iloveeclipse commented 2 days ago

"no screen devices" may relate to https://bugs.openjdk.org/browse/JDK-8336862

Great, isn't that the reason for https://github.com/eclipse-platform/eclipse.platform.swt/issues/1486 ?

So we should downgrade JDK on Windows to 17.0.11 version to get tests working again???

laeubi commented 2 days ago

Too bad we need AWT for SWT tests ...

So probably better use

final Display display = Display.getDefault();
final Image captureImage = new Image(display, display.getBounds().width, display.getBounds().height);
GC gc = new GC(display);
gc.copyArea(captureImage, display.getBounds().x, display.getBounds().y);
gc.dispose();
Imagewriter.write(...)

(see for example https://github.com/eclipse-swtbot/org.eclipse.swtbot/blob/ed2034a65915f6095d3c2464ac43068b105ebd68/org.eclipse.swtbot.swt.finder/src/org/eclipse/swtbot/swt/finder/utils/SWTUtils.java#L463 )