search

libsdl-org / SDL

Simple Directmedia Layer
https://libsdl.org
zlib License
10.19k stars 1.86k forks source link

testautomation fails on ppc64le #9219

Open bl4ckb0ne opened 8 months ago

bl4ckb0ne commented 8 months ago

I'm updating the Alpinlinux SDL2 package to 2.30.1 (2.30.0 has the same issue) and I have a single failure, on ppc64le.

INFO:  03/07/24 15:11:17: ----- Test Case 15.2: 'render_testPrimitives' started
INFO:  03/07/24 15:11:17: Test Description: 'Tests rendering primitives'
INFO:  03/07/24 15:11:17: Test Iteration 1: execKey 18148926758332540258
INFO:  03/07/24 15:11:17: Assert 'SDL_CreateWindow()': Passed
INFO:  03/07/24 15:11:17: Assert 'Check SDL_CreateWindow result': Passed
INFO: Created renderer: software
INFO:  03/07/24 15:11:17: Assert 'SDL_CreateRenderer()': Passed
INFO:  03/07/24 15:11:17: Assert 'Check SDL_CreateRenderer result': Passed
INFO:  03/07/24 15:11:17: Assert 'Validate result from SDL_SetRenderDrawColor, expected: 0, got: 0': Passed
INFO:  03/07/24 15:11:17: Assert 'Validate result from SDL_RenderClear, expected: 0, got: 0': Passed
INFO:  03/07/24 15:11:17: Assert 'Validate result from SDL_SetRenderDrawBlendMode, expected: 0, got: 0': Passed
INFO:  03/07/24 15:11:17: Assert 'Validate result from SDL_SetRenderDrawColor, expected: 0, got: 0': Passed
INFO:  03/07/24 15:11:17: Assert '_hasDrawColor': Passed
INFO:  03/07/24 15:11:17: Assert 'Validate result from SDL_SetRenderDrawColor, expected: 0, got: 0': Passed
INFO:  03/07/24 15:11:17: Assert 'Validate result from SDL_RenderFillRect, expected: 0, got: 0': Passed
INFO:  03/07/24 15:11:17: Assert 'Validate result from SDL_SetRenderDrawColor, expected: 0, got: 0': Passed
INFO:  03/07/24 15:11:17: Assert 'Validate result from SDL_RenderFillRect, expected: 0, got: 0': Passed
INFO:  03/07/24 15:11:17: Assert 'Validate results from calls to SDL_SetRenderDrawColor, expected: 0, got: 0': Passed
INFO:  03/07/24 15:11:17: Assert 'Validate results from calls to SDL_RenderDrawPoint, expected: 0, got: 0': Passed
INFO:  03/07/24 15:11:17: Assert 'Validate result from SDL_SetRenderDrawColor': Passed
INFO:  03/07/24 15:11:17: Assert 'Validate result from SDL_RenderDrawLine, expected: 0, got: 0': Passed
INFO:  03/07/24 15:11:17: Assert 'Validate result from SDL_SetRenderDrawColor, expected: 0, got: 0': Passed
INFO:  03/07/24 15:11:17: Assert 'Validate result from SDL_RenderDrawLine, expected: 0, got: 0': Passed
INFO:  03/07/24 15:11:17: Assert 'Validate result from SDL_SetRenderDrawColor, expected: 0, got: 0': Passed
INFO:  03/07/24 15:11:17: Assert 'Validate result from SDL_RenderDrawLine, expected: 0, got: 0': Passed
INFO:  03/07/24 15:11:17: Assert 'Validate result from SDL_RenderDrawLine, expected: 0, got: 0': Passed
INFO:  03/07/24 15:11:17: Assert 'Validate result from SDL_RenderDrawLine, expected: 0, got: 0': Passed
INFO:  03/07/24 15:11:17: Assert 'Validate result from SDL_RenderDrawLine, expected: 0, got: 0': Passed
INFO:  03/07/24 15:11:17: Assert 'Validate allocated temp pixel buffer': Passed
INFO:  03/07/24 15:11:17: Assert 'Validate result from SDL_RenderReadPixels, expected: 0, got: 0': Passed
INFO:  03/07/24 15:11:17: Assert 'Verify result from SDL_CreateRGBSurfaceFrom is not NULL': Passed
ERROR: 03/07/24 15:11:17: Comparison of pixels with allowable error of 0 failed 4800 times.
ERROR: 03/07/24 15:11:17: First detected occurrence at position 0,0 with a squared RGB-difference of 83525.
ERROR: 03/07/24 15:11:17: Surfaces from failed comparison saved as 'CompareSurfaces0001_TestOutput.bmp' and 'CompareSurfaces0001_Reference.bmp'
ERROR: 03/07/24 15:11:17: Assert 'Validate result from SDLTest_CompareSurfaces, expected: 0, got: 4800': Failed
INFO:  03/07/24 15:11:17: Assert 'SDL_DestroyRenderer()': Passed
INFO:  03/07/24 15:11:17: Assert 'SDL_DestroyWindow': Passed
ERROR: 03/07/24 15:11:17: Assert Summary: Total=30 Passed=29 Failed=1
INFO:  03/07/24 15:11:17: Total Test runtime: 0.0 sec
ERROR: 03/07/24 15:11:17: >>> Test 'render_testPrimitives': Failed
INFO:  03/07/24 15:11:17: ----- Test Case 15.3: 'render_testPrimitivesBlend' started
INFO:  03/07/24 15:11:17: Test Description: 'Tests rendering primitives with blending'
INFO:  03/07/24 15:11:17: Test Iteration 1: execKey 5052025182091858433
INFO:  03/07/24 15:11:17: >>> Test 'render_testPrimitivesBlend': Skipped (Disabled)
INFO:  03/07/24 15:11:17: Total Test runtime: 0.0 sec
INFO:  03/07/24 15:11:17: ----- Test Case 15.4: 'render_testBlit' started
INFO:  03/07/24 15:11:17: Test Description: 'Tests blitting'
INFO:  03/07/24 15:11:17: Test Iteration 1: execKey 14490451536073541954
INFO:  03/07/24 15:11:17: Assert 'SDL_CreateWindow()': Passed
INFO:  03/07/24 15:11:17: Assert 'Check SDL_CreateWindow result': Passed
INFO: Created renderer: software
INFO:  03/07/24 15:11:17: Assert 'SDL_CreateRenderer()': Passed
INFO:  03/07/24 15:11:17: Assert 'Check SDL_CreateRenderer result': Passed
INFO:  03/07/24 15:11:17: Assert 'Validate result from SDL_SetRenderDrawColor, expected: 0, got: 0': Passed
INFO:  03/07/24 15:11:17: Assert 'Validate result from SDL_RenderClear, expected: 0, got: 0': Passed
INFO:  03/07/24 15:11:17: Assert 'Validate result from SDL_SetRenderDrawBlendMode, expected: 0, got: 0': Passed
INFO:  03/07/24 15:11:17: Assert 'Validate result from SDL_SetRenderDrawColor, expected: 0, got: 0': Passed
INFO:  03/07/24 15:11:17: Assert '_hasDrawColor)': Passed
INFO:  03/07/24 15:11:17: Assert 'Verify _loadTestFace() result': Passed
INFO:  03/07/24 15:11:17: Assert 'Verify result from SDL_QueryTexture, expected 0, got 0': Passed
INFO:  03/07/24 15:11:17: Assert 'Validate results from calls to SDL_RenderCopy, expected: 0, got: 0': Passed
INFO:  03/07/24 15:11:17: Assert 'Validate allocated temp pixel buffer': Passed
INFO:  03/07/24 15:11:17: Assert 'Validate result from SDL_RenderReadPixels, expected: 0, got: 0': Passed
INFO:  03/07/24 15:11:17: Assert 'Verify result from SDL_CreateRGBSurfaceFrom is not NULL': Passed
ERROR: 03/07/24 15:11:17: Comparison of pixels with allowable error of 0 failed 4800 times.
ERROR: 03/07/24 15:11:17: First detected occurrence at position 0,0 with a squared RGB-difference of 65025.
ERROR: 03/07/24 15:11:17: Surfaces from failed comparison saved as 'CompareSurfaces0002_TestOutput.bmp' and 'CompareSurfaces0002_Reference.bmp'
ERROR: 03/07/24 15:11:17: Assert 'Validate result from SDLTest_CompareSurfaces, expected: 0, got: 4800': Failed
INFO:  03/07/24 15:11:17: Assert 'SDL_DestroyRenderer()': Passed
INFO:  03/07/24 15:11:17: Assert 'SDL_DestroyWindow': Passed
ERROR: 03/07/24 15:11:17: Assert Summary: Total=18 Passed=17 Failed=1
INFO:  03/07/24 15:11:17: Total Test runtime: 0.0 sec
ERROR: 03/07/24 15:11:17: >>> Test 'render_testBlit': Failed
INFO:  03/07/24 15:11:17: ----- Test Case 15.5: 'render_testBlitColor' started
INFO:  03/07/24 15:11:17: Test Description: 'Tests blitting with color'
INFO:  03/07/24 15:11:17: Test Iteration 1: execKey 16161055527516166808
INFO:  03/07/24 15:11:17: Assert 'SDL_CreateWindow()': Passed
INFO:  03/07/24 15:11:17: Assert 'Check SDL_CreateWindow result': Passed
INFO: Created renderer: software
INFO:  03/07/24 15:11:17: Assert 'SDL_CreateRenderer()': Passed
INFO:  03/07/24 15:11:17: Assert 'Check SDL_CreateRenderer result': Passed
INFO:  03/07/24 15:11:17: Assert 'Validate result from SDL_SetRenderDrawColor, expected: 0, got: 0': Passed
INFO:  03/07/24 15:11:17: Assert 'Validate result from SDL_RenderClear, expected: 0, got: 0': Passed
INFO:  03/07/24 15:11:17: Assert 'Validate result from SDL_SetRenderDrawBlendMode, expected: 0, got: 0': Passed
INFO:  03/07/24 15:11:17: Assert 'Validate result from SDL_SetRenderDrawColor, expected: 0, got: 0': Passed
INFO:  03/07/24 15:11:17: Assert 'Verify _loadTestFace() result': Passed
INFO:  03/07/24 15:11:17: Assert 'Verify result from SDL_QueryTexture, expected 0, got 0': Passed
INFO:  03/07/24 15:11:17: Assert 'Validate results from calls to SDL_SetTextureColorMod, expected: 0, got: 0': Passed
INFO:  03/07/24 15:11:17: Assert 'Validate results from calls to SDL_RenderCopy, expected: 0, got: 0': Passed
INFO:  03/07/24 15:11:17: Assert 'Validate allocated temp pixel buffer': Passed
INFO:  03/07/24 15:11:17: Assert 'Validate result from SDL_RenderReadPixels, expected: 0, got: 0': Passed
INFO:  03/07/24 15:11:17: Assert 'Verify result from SDL_CreateRGBSurfaceFrom is not NULL': Passed
ERROR: 03/07/24 15:11:17: Comparison of pixels with allowable error of 0 failed 4800 times.
ERROR: 03/07/24 15:11:17: First detected occurrence at position 0,0 with a squared RGB-difference of 65025.
ERROR: 03/07/24 15:11:17: Surfaces from failed comparison saved as 'CompareSurfaces0003_TestOutput.bmp' and 'CompareSurfaces0003_Reference.bmp'
ERROR: 03/07/24 15:11:17: Assert 'Validate result from SDLTest_CompareSurfaces, expected: 0, got: 4800': Failed
INFO:  03/07/24 15:11:17: Assert 'SDL_DestroyRenderer()': Passed
INFO:  03/07/24 15:11:17: Assert 'SDL_DestroyWindow': Passed
ERROR: 03/07/24 15:11:17: Assert Summary: Total=18 Passed=17 Failed=1
INFO:  03/07/24 15:11:17: Total Test runtime: 0.0 sec
ERROR: 03/07/24 15:11:17: >>> Test 'render_testBlitColor': Failed

I'm working on pulling the output images from the CI, will update the issue later.

Thanks

madebr commented 8 months ago

Quoted from https://github.com/libsdl-org/SDL/issues/9365#issuecomment-2026134173

Still failing but on ppc64le only 1 but I think i already reported it here #9219

Interesting to see it failing for testautomation, but not for testautomation-no-simd. So do we have an issue with AltiVec simd?

madebr commented 3 weeks ago

@bl4ckb0ne Does this still happen on ppc64le? What about testautomation <-> testautomation-no-simd?

bl4ckb0ne commented 3 weeks ago

@madebr I reenabled the test in https://gitlab.alpinelinux.org/alpine/aports/-/merge_requests/74574 with the latest release, we'll see how it goes.

bl4ckb0ne commented 3 weeks ago

testautomation still failing https://gitlab.alpinelinux.org/alpine/aports/-/jobs/1587325