JetBrains / skiko

Kotlin Multiplatform bindings to Skia
Apache License 2.0
1.81k stars 111 forks source link

Mali-400 OpenGL ES2.0 Linux arm64 #481

Closed basicasicmatrix closed 1 week ago

basicasicmatrix commented 2 years ago

Hello,

Is skiko compatible with Mali-400 OpenGL ES 2.0 - Linux arm64?

I'm running into the following errors:

Exception in thread "AWT-EventQueue-0" java.lang.IllegalArgumentException: Failed Surface.makeFromBackendRenderTarget
    at org.jetbrains.skia.Surface$Companion.makeFromBackendRenderTarget(Surface.kt:287)
    at org.jetbrains.skia.Surface$Companion.makeFromBackendRenderTarget$default(Surface.kt:269)
    at org.jetbrains.skiko.context.OpenGLContextHandler.initCanvas(OpenGLContextHandler.kt:57)
    at org.jetbrains.skiko.context.ContextHandler.draw(ContextHandler.kt:49)
    at org.jetbrains.skiko.redrawer.LinuxOpenGLRedrawer.draw(LinuxOpenGLRedrawer.kt:90)
    at org.jetbrains.skiko.redrawer.LinuxOpenGLRedrawer.redrawImmediately(LinuxOpenGLRedrawer.kt:78)
    at org.jetbrains.skiko.SkiaLayer.paint(SkiaLayer.jvm.kt:324)
    at androidx.compose.ui.awt.ComposeLayer$ComponentImpl.paint(ComposeLayer.desktop.kt:144)
    at java.desktop/javax.swing.JComponent.paintChildren(JComponent.java:952)
    at java.desktop/javax.swing.JComponent.paint(JComponent.java:1128)
    at java.desktop/javax.swing.JLayeredPane.paint(JLayeredPane.java:586)
    at java.desktop/javax.swing.JComponent.paintChildren(JComponent.java:952)
    at java.desktop/javax.swing.JComponent.paint(JComponent.java:1128)
    at java.desktop/javax.swing.JComponent.paintToOffscreen(JComponent.java:5311)
    at java.desktop/javax.swing.BufferStrategyPaintManager.paint(BufferStrategyPaintManager.java:246)
    at java.desktop/javax.swing.RepaintManager.paint(RepaintManager.java:1337)
    at java.desktop/javax.swing.JComponent._paintImmediately(JComponent.java:5259)
    at java.desktop/javax.swing.JComponent.paintImmediately(JComponent.java:5069)
    at java.desktop/javax.swing.RepaintManager$4.run(RepaintManager.java:879)
    at java.desktop/javax.swing.RepaintManager$4.run(RepaintManager.java:862)
    at java.base/java.security.AccessController.doPrivileged(AccessController.java:399)
    at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:86)
    at java.desktop/javax.swing.RepaintManager.paintDirtyRegions(RepaintManager.java:862)
    at java.desktop/javax.swing.RepaintManager.paintDirtyRegions(RepaintManager.java:835)
    at java.desktop/javax.swing.RepaintManager.prePaintDirtyRegions(RepaintManager.java:784)
    at java.desktop/javax.swing.RepaintManager$ProcessingRunnable.run(RepaintManager.java:1898)
    at java.desktop/java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:318)
    at java.desktop/java.awt.EventQueue.dispatchEventImpl(EventQueue.java:771)
    at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:722)
    at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:716)
    at java.base/java.security.AccessController.doPrivileged(AccessController.java:399)
    at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:86)
    at java.desktop/java.awt.EventQueue.dispatchEvent(EventQueue.java:741)
    at java.desktop/java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:203)
    at java.desktop/java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:124)
    at java.desktop/java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:113)
    at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:109)
    at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
    at java.desktop/java.awt.EventDispatchThread.run(EventDispatchThread.java:90)
Exception in thread "AWT-EventQueue-0" java.lang.IllegalArgumentException: Failed Surface.makeFromBackendRenderTarget
    at org.jetbrains.skia.Surface$Companion.makeFromBackendRenderTarget(Surface.kt:287)
    at org.jetbrains.skia.Surface$Companion.makeFromBackendRenderTarget$default(Surface.kt:269)
    at org.jetbrains.skiko.context.OpenGLContextHandler.initCanvas(OpenGLContextHandler.kt:57)
    at org.jetbrains.skiko.context.ContextHandler.draw(ContextHandler.kt:49)
    at org.jetbrains.skiko.redrawer.LinuxOpenGLRedrawer.draw(LinuxOpenGLRedrawer.kt:90)
    at org.jetbrains.skiko.redrawer.LinuxOpenGLRedrawer.access$draw(LinuxOpenGLRedrawer.kt:9)
    at org.jetbrains.skiko.redrawer.LinuxOpenGLRedrawer$Companion$frameDispatcher$1.invokeSuspend(LinuxOpenGLRedrawer.kt:122)
    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
    at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
    at java.desktop/java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:318)
    at java.desktop/java.awt.EventQueue.dispatchEventImpl(EventQueue.java:771)
    at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:722)
    at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:716)
    at java.base/java.security.AccessController.doPrivileged(AccessController.java:399)
    at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:86)
    at java.desktop/java.awt.EventQueue.dispatchEvent(EventQueue.java:741)
    at java.desktop/java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:203)
    at java.desktop/java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:124)
    at java.desktop/java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:113)
    at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:109)
    at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
    at java.desktop/java.awt.EventDispatchThread.run(EventDispatchThread.java:90)
olonho commented 2 years ago

Which version of Skiko is it? Could you try to extract more logs?

basicasicmatrix commented 2 years ago

@olonho

org.jetbrains.skiko:skiko-jvm-runtime-linux-arm64:0.6.6 using repositories [MavenRepo, maven, Google, BintrayJCenter]
2022-02-07T18:03:11.838+0000 [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.ivyresolve.CachingModuleComponentRepository]

It's being imported from JetBrains/compose-jb.

I've conducted an apitrace to see which OpenGL or OpenGL ES functions have been called, and got the following:

gl

0 glXChooseVisual(dpy = 0x7fb81cf230, screen = 0, attribList = {GLX_RGBA, GLX_DOUBLEBUFFER, 1, 0}) = &{visual = 0x7fb81bf420, visualid = 972, screen = 0, depth = 24, c_class = 4, red_mask = 16711680, green_mask = 65280, blue_mask = 255, colormap_size = 256, bits_per_rgb = 8}
1 glXCreateContext(dpy = 0x7fb81cf230, vis = &{visual = 0x7fb81bf420, visualid = 972, screen = 0, depth = 24, c_class = 4, red_mask = 16711680, green_mask = 65280, blue_mask = 255, colormap_size = 256, bits_per_rgb = 8}, shareList = NULL, direct = True) = 0x7f402bdb70
2 glXMakeCurrent(dpy = 0x7fb81cf230, drawable = 8388640, ctx = 0x7f402bdb70) = True
3 glViewport(x = 0, y = 0, width = 1, height = 1) // fake
4 glScissor(x = 0, y = 0, width = 1, height = 1) // fake
7 glXSwapIntervalEXT(dpy = 0x7fb81cf230, drawable = 8388640, interval = 1)
8 glXMakeCurrent(dpy = 0x7fb81cf230, drawable = 8388640, ctx = 0x7f402bdb70) = True
172 glGetIntegerv(pname = GL_MAX_FRAGMENT_UNIFORM_COMPONENTS, params = &16384)
173 glGetIntegerv(pname = GL_MAX_VERTEX_ATTRIBS, params = &16)
174 glGetShaderPrecisionFormat(shadertype = GL_FRAGMENT_SHADER, precisiontype = GL_HIGH_FLOAT, range = {127, 127}, precision = {23, 1081738920})
175 glGetShaderPrecisionFormat(shadertype = GL_VERTEX_SHADER, precisiontype = GL_HIGH_FLOAT, range = {127, 127}, precision = {23, 1081738920})
176 glGetShaderPrecisionFormat(shadertype = GL_FRAGMENT_SHADER, precisiontype = GL_MEDIUM_FLOAT, range = {127, 127}, precision = {23, 1081738920})
177 glGetShaderPrecisionFormat(shadertype = GL_VERTEX_SHADER, precisiontype = GL_MEDIUM_FLOAT, range = {127, 127}, precision = {23, 1081738920})
178 glGetIntegerv(pname = GL_MAX_TEXTURE_IMAGE_UNITS, params = &16)
179 glGetIntegerv(pname = GL_MAX_TEXTURE_SIZE, params = &4096)
180 glGetIntegerv(pname = GL_MAX_RENDERBUFFER_SIZE, params = &4096)
181 glGetInternalformativ(target = GL_RENDERBUFFER, internalformat = GL_RGBA8, pname = GL_NUM_SAMPLE_COUNTS, bufSize = 1, params = &0)
182 glGetInternalformativ(target = GL_RENDERBUFFER, internalformat = GL_RGB565, pname = GL_NUM_SAMPLE_COUNTS, bufSize = 1, params = &0)
183 glGetInternalformativ(target = GL_RENDERBUFFER, internalformat = GL_RGB10_A2, pname = GL_NUM_SAMPLE_COUNTS, bufSize = 1, params = &0)
184 glGetInternalformativ(target = GL_RENDERBUFFER, internalformat = GL_SRGB8_ALPHA8, pname = GL_NUM_SAMPLE_COUNTS, bufSize = 1, params = &0)
186 glGetIntegerv(pname = GL_DRAW_FRAMEBUFFER_BINDING, params = &0)
187 glXMakeCurrent(dpy = 0x7fb81cf230, drawable = 8388640, ctx = 0x7f402bdb70) = True
188 glGetIntegerv(pname = GL_DRAW_FRAMEBUFFER_BINDING, params = &0)

egl

// [process.name](http://process.name/) = "/usr/lib/jvm/java-17-openjdk-arm64/bin/java"
1 glGetIntegerv(pname = GL_DRAW_FRAMEBUFFER_BINDING, params = &0)
2 glGetIntegerv(pname = GL_DRAW_FRAMEBUFFER_BINDING, params = &0)

OS is Mobian on a PinePhone.

olonho commented 2 years ago

Cold you try 0.7.7, which is the current version.

sebkur commented 2 years ago

when trying this using a compose for desktop app, do you think it could work by still using id("org.jetbrains.compose") version "1.0.1" but swapping out org.jetbrains.skiko:skiko:0.6.6 with org.jetbrains.skiko:skiko:0.7.7? Or will that not replace enough dependencies / shared native libraries?

sebkur commented 2 years ago

or maybe it makes more sense to use a more recent version of compose, i.e. 1.2.0-alpha01-dev606, however that still depends on org.jetbrains.skiko:skiko:0.7.3?

basicasicmatrix commented 2 years ago

Cold you try 0.7.7, which is the current version.

Tried 0.7.3 and 0.7.7 and getting the same initial error.

olonho commented 2 years ago

Then we probably need more Skiko debugging (you don’t need Compose, just use Skiko demo application, error shall be the same). My hypothesis is some color depth/gl config mismatch, but without hardware really hard to guess.

olonho commented 2 years ago

See https://github.com/JetBrains/skiko/tree/master/samples/SkiaAwtSample

sebkur commented 2 years ago

ah, great a sample without compose using skia directly! Less abstraction, helpful for debugging

basicasicmatrix commented 2 years ago

Then we probably need more Skiko debugging (you don’t need Compose, just use Skiko demo application, error shall be the same). My hypothesis is some color depth/gl config mismatch, but without hardware really hard to guess.

As expected, the error is the same.

> Task :run
Changed renderer for org.jetbrains.skiko.SkiaLayer[,0,0,0x0,invalid,alignmentX=0.0,alignmentY=0.0,border=,flags=16777217,maximumSize=,minimumSize=,preferredSize=]: new value is OPENGL
[20382761] FPS 0 (0-0)
GraphicsApi: OPENGL
OS: linux arm64
Vendor: lima
Model: Mali400
Total VRAM: 0 MB

Exception in thread "AWT-EventQueue-0" java.lang.IllegalArgumentException: Failed Surface.makeFromBackendRenderTarget
    at org.jetbrains.skia.Surface$Companion.makeFromBackendRenderTarget(Surface.kt:287)
    at org.jetbrains.skia.Surface$Companion.makeFromBackendRenderTarget$default(Surface.kt:269)
    at org.jetbrains.skiko.context.OpenGLContextHandler.initCanvas(OpenGLContextHandler.kt:57)
    at org.jetbrains.skiko.context.ContextHandler.draw(ContextHandler.kt:49)
    at org.jetbrains.skiko.redrawer.LinuxOpenGLRedrawer.redrawImmediately(LinuxOpenGLRedrawer.kt:77)
    at org.jetbrains.skiko.SkiaLayer.paint(SkiaLayer.awt.kt:340)
    at SkiaAwtSample.AppKt.createWindow$lambda-1(App.kt:117)
    at java.desktop/java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:318)
    at java.desktop/java.awt.EventQueue.dispatchEventImpl(EventQueue.java:770)
    at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:721)
    at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:715)
    at java.base/java.security.AccessController.doPrivileged(AccessController.java:391)
    at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:85)
    at java.desktop/java.awt.EventQueue.dispatchEvent(EventQueue.java:740)
    at java.desktop/java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:203)
    at java.desktop/java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:124)
    at java.desktop/java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:113)
    at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:109)
    at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
    at java.desktop/java.awt.EventDispatchThread.run(EventDispatchThread.java:90)

Is there some specific debugging steps you can recommend?

olonho commented 2 years ago

Could you please try to use in https://github.com/JetBrains/skiko/blob/99eeba5417037e341f78c91cec2bedb764d38c14/skiko/src/awtMain/kotlin/org/jetbrains/skiko/context/OpenGLContextHandler.kt#L55

some other SurfaceColorFormat, as defined in

https://github.com/JetBrains/skiko/blob/99eeba5417037e341f78c91cec2bedb764d38c14/skiko/src/commonMain/kotlin/org/jetbrains/skia/SurfaceColorFormat.kt#L4

olonho commented 2 years ago

Also you may try to use SKIKO_RENDER_API=SOFTWARE and report if it works.

olonho commented 2 years ago

Also using 0 instead of 8 as stencilBits parameter in https://github.com/JetBrains/skiko/blob/99eeba5417037e341f78c91cec2bedb764d38c14/skiko/src/awtMain/kotlin/org/jetbrains/skiko/context/OpenGLContextHandler.kt#L55 may help as well.

basicasicmatrix commented 2 years ago

@olonho

Tried all the SurfaceColorFormats, and several with 0 as stencilBits parameter - all to the effect of same original error. SKIKO_RENDER_API=SOFTWARE does work on the test, unfortunately froze the hardware on test in our end application - seems we need the hardware acceleration.

Let me know if you think of anything else.

olonho commented 2 years ago

Software rendering must work, maybe report/debug on how it hangs would be cool. For hardware rendering you need to debug, why makeGLRenderTarget fails, maybe by debugging Skiko. To do that you may want to rebuild Skiko with smth like ./gradlew -Pskiko.debug=true publishToMavenLocal

Rsedaikin commented 2 years ago

@basicasicmatrix Could you provide information about your device you are working on? Model for example. Thank you.

basicasicmatrix commented 2 years ago

Software rendering must work, maybe report/debug on how it hangs would be cool. For hardware rendering you need to debug, why makeGLRenderTarget fails, maybe by debugging Skiko. To do that you may want to rebuild Skiko with smth like ./gradlew -Pskiko.debug=true publishToMavenLocal

It turns out the device was overheating from all the compile tasks, after it cooled down Software rendering functioned as expected. I’ll try again with hardware with the debug flag you’ve suggested.

@basicasicmatrix Could you provide information about your device you are working on? Model for example. Thank you.

Device is the PinePhone (original, not Pro). https://wiki.pine64.org/index.php/PinePhone

olonho commented 2 years ago

Force using GLES on Linux/arm64 builds, will be available with next milestone build of skia-pack https://github.com/JetBrains/skia-pack/commit/453c3e736079abc47a89f9ac6c6bdb24dc6823e6 and also implemented better fallback routine in https://github.com/JetBrains/skiko/pull/487

olonho commented 2 years ago

Please try Skiko 0.7.9, at least fallback routine shall be fixed (acceleration will not yet).

okushnikov commented 1 week ago

Please check the following ticket on YouTrack for follow-ups to this issue. GitHub issues will be closed in the coming weeks.