search

KhronosGroup / VK-GL-CTS

Khronos Vulkan, OpenGL, and OpenGL ES Conformance Tests
https://www.khronos.org/
Apache License 2.0
525 stars 293 forks source link

[bisected] EGL.functional.color_clears.*other on Wayland failling since 1.3.7.0 on almost everything #433

Open okias opened 10 months ago

okias commented 10 months ago

Problem:

src/freedreno/ci/freedreno-a630-fails.txt
616:wayland-dEQP-EGL.functional.color_clears.multi_context.gles1.other,Fail
617:wayland-dEQP-EGL.functional.color_clears.multi_context.gles1_gles2.other,Fail
618:wayland-dEQP-EGL.functional.color_clears.multi_context.gles1_gles2_gles3.other,Fail
619:wayland-dEQP-EGL.functional.color_clears.multi_context.gles2.other,Fail
620:wayland-dEQP-EGL.functional.color_clears.multi_context.gles3.other,Fail
621:wayland-dEQP-EGL.functional.color_clears.multi_thread.gles1.other,Fail
622:wayland-dEQP-EGL.functional.color_clears.multi_thread.gles1_gles2.other,Fail
623:wayland-dEQP-EGL.functional.color_clears.multi_thread.gles1_gles2_gles3.other,Fail
624:wayland-dEQP-EGL.functional.color_clears.multi_thread.gles2.other,Fail
625:wayland-dEQP-EGL.functional.color_clears.multi_thread.gles3.other,Fail
626:wayland-dEQP-EGL.functional.color_clears.single_context.gles1.other,Fail
627:wayland-dEQP-EGL.functional.color_clears.single_context.gles2.other,Fail
628:wayland-dEQP-EGL.functional.color_clears.single_context.gles3.other,Fail

src/freedreno/ci/freedreno-a618-fails.txt
676:wayland-dEQP-EGL.functional.color_clears.multi_context.gles1.other,Fail
677:wayland-dEQP-EGL.functional.color_clears.multi_context.gles1_gles2.other,Fail
678:wayland-dEQP-EGL.functional.color_clears.multi_context.gles1_gles2_gles3.other,Fail
679:wayland-dEQP-EGL.functional.color_clears.multi_context.gles2.other,Fail
680:wayland-dEQP-EGL.functional.color_clears.multi_context.gles3.other,Fail
681:wayland-dEQP-EGL.functional.color_clears.multi_thread.gles1.other,Fail
682:wayland-dEQP-EGL.functional.color_clears.multi_thread.gles1_gles2.other,Fail
683:wayland-dEQP-EGL.functional.color_clears.multi_thread.gles1_gles2_gles3.other,Fail
684:wayland-dEQP-EGL.functional.color_clears.multi_thread.gles2.other,Fail
685:wayland-dEQP-EGL.functional.color_clears.multi_thread.gles3.other,Fail
686:wayland-dEQP-EGL.functional.color_clears.single_context.gles1.other,Fail
687:wayland-dEQP-EGL.functional.color_clears.single_context.gles2.other,Fail
688:wayland-dEQP-EGL.functional.color_clears.single_context.gles3.other,Fail

src/intel/ci/iris-glk-fails.txt
7:wayland-dEQP-EGL.functional.color_clears.multi_context.gles1.other,Fail
8:wayland-dEQP-EGL.functional.color_clears.multi_context.gles1_gles2.other,Fail
9:wayland-dEQP-EGL.functional.color_clears.multi_context.gles1_gles2_gles3.other,Fail
10:wayland-dEQP-EGL.functional.color_clears.multi_context.gles2.other,Fail
11:wayland-dEQP-EGL.functional.color_clears.multi_context.gles3.other,Fail
12:wayland-dEQP-EGL.functional.color_clears.multi_thread.gles1.other,Fail
13:wayland-dEQP-EGL.functional.color_clears.multi_thread.gles1_gles2.other,Fail
14:wayland-dEQP-EGL.functional.color_clears.multi_thread.gles1_gles2_gles3.other,Fail
15:wayland-dEQP-EGL.functional.color_clears.multi_thread.gles2.other,Fail
16:wayland-dEQP-EGL.functional.color_clears.multi_thread.gles3.other,Fail
17:wayland-dEQP-EGL.functional.color_clears.single_context.gles1.other,Fail
18:wayland-dEQP-EGL.functional.color_clears.single_context.gles2.other,Fail
19:wayland-dEQP-EGL.functional.color_clears.single_context.gles3.other,Fail

src/intel/ci/iris-apl-fails.txt
2:wayland-dEQP-EGL.functional.color_clears.multi_context.gles1.other,Fail
3:wayland-dEQP-EGL.functional.color_clears.multi_context.gles1_gles2.other,Fail
4:wayland-dEQP-EGL.functional.color_clears.multi_context.gles1_gles2_gles3.other,Fail
5:wayland-dEQP-EGL.functional.color_clears.multi_context.gles2.other,Fail
6:wayland-dEQP-EGL.functional.color_clears.multi_context.gles3.other,Fail
7:wayland-dEQP-EGL.functional.color_clears.multi_thread.gles1.other,Fail
8:wayland-dEQP-EGL.functional.color_clears.multi_thread.gles1_gles2.other,Fail
9:wayland-dEQP-EGL.functional.color_clears.multi_thread.gles1_gles2_gles3.other,Fail
10:wayland-dEQP-EGL.functional.color_clears.multi_thread.gles2.other,Fail
11:wayland-dEQP-EGL.functional.color_clears.multi_thread.gles3.other,Fail
12:wayland-dEQP-EGL.functional.color_clears.single_context.gles1.other,Fail
13:wayland-dEQP-EGL.functional.color_clears.single_context.gles2.other,Fail
14:wayland-dEQP-EGL.functional.color_clears.single_context.gles3.other,Fail

src/broadcom/ci/broadcom-rpi4-fails.txt
531:wayland-dEQP-EGL.functional.color_clears.multi_context.gles1.other,Fail
532:wayland-dEQP-EGL.functional.color_clears.multi_context.gles1_gles2.other,Fail
533:wayland-dEQP-EGL.functional.color_clears.multi_context.gles1_gles2_gles3.other,Fail
534:wayland-dEQP-EGL.functional.color_clears.multi_context.gles2.other,Fail
535:wayland-dEQP-EGL.functional.color_clears.multi_context.gles3.other,Fail
536:wayland-dEQP-EGL.functional.color_clears.multi_thread.gles1.other,Fail
537:wayland-dEQP-EGL.functional.color_clears.multi_thread.gles1_gles2.other,Fail
538:wayland-dEQP-EGL.functional.color_clears.multi_thread.gles1_gles2_gles3.other,Fail
539:wayland-dEQP-EGL.functional.color_clears.multi_thread.gles2.other,Fail
540:wayland-dEQP-EGL.functional.color_clears.multi_thread.gles3.other,Fail
541:wayland-dEQP-EGL.functional.color_clears.single_context.gles1.other,Fail
542:wayland-dEQP-EGL.functional.color_clears.single_context.gles2.other,Fail
543:wayland-dEQP-EGL.functional.color_clears.single_context.gles3.other,Fail

src/panfrost/ci/panfrost-t860-fails.txt
60:wayland-dEQP-EGL.functional.color_clears.multi_context.gles1.other,Fail
61:wayland-dEQP-EGL.functional.color_clears.multi_context.gles1_gles2.other,Fail
62:wayland-dEQP-EGL.functional.color_clears.multi_context.gles1_gles2_gles3.other,Fail
63:wayland-dEQP-EGL.functional.color_clears.multi_context.gles2.other,Fail
64:wayland-dEQP-EGL.functional.color_clears.multi_context.gles3.other,Fail
65:wayland-dEQP-EGL.functional.color_clears.multi_thread.gles1.other,Fail
66:wayland-dEQP-EGL.functional.color_clears.multi_thread.gles1_gles2.other,Fail
67:wayland-dEQP-EGL.functional.color_clears.multi_thread.gles1_gles2_gles3.other,Fail
68:wayland-dEQP-EGL.functional.color_clears.multi_thread.gles2.other,Fail
69:wayland-dEQP-EGL.functional.color_clears.multi_thread.gles3.other,Fail
70:wayland-dEQP-EGL.functional.color_clears.single_context.gles1.other,Fail
71:wayland-dEQP-EGL.functional.color_clears.single_context.gles2.other,Fail
72:wayland-dEQP-EGL.functional.color_clears.single_context.gles3.other,Fail

Bisected to the 79b25659bcbced0cfc2c3fe318951c585f682abe

Fix of egl surface creating failure in RenderCase

Get the EGL_SURFACE_TYPE of current egl config, if some surface type is not supported, skip its excute block.

Affects:
dEQP-EGL.functional.query_context.*.other

VK-GL-CTS issue: 4512
Components: EGL

Change-Id: Ie2355770f78783cb47b5a15bf7f35e26f62a4cfd

Since the Affects: doesn't mention color_clears, not sure if a bug or feature.

infapi00 commented 10 months ago

Since the Affects: doesn't mention color_clears, not sure if a bug or feature.

One thing that we noted recently (although we didn't bisect to that point) is that several EGL (and some GLES tests) now try to use RB10A0 and RGB10A2 formats, and several tests were not handling that correctly. Specifically ...

src/broadcom/ci/broadcom-rpi4-fails.txt

... those tests that you list there were failing for our case due that. During the past weeks several developers have been providing patches related to those formats, like for example:

https://github.com/KhronosGroup/VK-GL-CTS/commit/d7905ab6f4e3de817e5269f23461b0be197fb16d

Most of them are already on the main branch or at least on an opengl-es branch. So there are a fix for those tests.

But about the tags. Those fixes are not on the 1.3.7.3 tag, and I think that they will arrive first to the opengl-es-cts branches, as they belong there. This is something that I asked @jasuarez some time about about the Mesa CI, that it would make more sense to test the vulkan tests with a vulkan tag, and the gles/egl tests with a opengl-es tag, but that would make things more complex on the CI.

In other words, the fixes for the patches are there, so it would make sense to close the issue, but on the other side, it is true that they are not yet on a cts tag, and it is not clear when they will be cherry picked.

@rg3igalia do you know who we should CC to clarify my previous doubt?

rg3igalia commented 10 months ago

In other words, the fixes for the patches are there, so it would make sense to close the issue, but on the other side, it is true that they are not yet on a cts tag, and it is not clear when they will be cherry picked.

@rg3igalia do you know who we should CC to clarify my previous doubt?

Since I normally don't work on OpenGL(ES) tests, I'm unsure about the policy regarding those branches. However, I see that recent and old-ish fixes for opengl-es-cts-3.2.9, while not present in most Vulkan branches, are present in main. This means they will eventually land in a vulkan-cts branch, indirectly, when it's forked to create a new stable release.

Pinging @debater and/or @lordalcol for confirmation.

that it would make more sense to test the vulkan tests with a vulkan tag, and the gles/egl tests with a opengl-es tag, but that would make things more complex on the CI.

That's a bit unfortunate, but I think it would make sense.

lordalcol commented 10 months ago

You are correct. The fixes will be part of the next release vulkan-cts-1.3.8.0, currently planned for end of January. We are currently working on other fixes for OpenGL(ES) but we do not currently have a planned release for that.

infapi00 commented 10 months ago

You are correct. The fixes will be part of the next release vulkan-cts-1.3.8.0, currently planned for end of January. We are currently working on other fixes for OpenGL(ES) but we do not currently have a planned release for that.

Thanks for confirming. So how vk-gl-cts handle this kind of issues? Should it still be open until that tag is released (so we can confirm if all those patches fixes those tests), or it would be better to close the issue as there is available a fix, that we know that fix those tests on one of those platforms?

lordalcol commented 10 months ago

Let's keep it open until it's released at least as part of a Vulkan tag.