Sometimes sdl2 - UnicodeDecodeError: 'utf-8' codec can't decode byte 0x98

[ghost]

@algorys commented on May 6, 2018, 8:06 AM UTC:

Versions

• Python: 3.5.2
• OS: Ubuntu 16.04 LTS
• Kivy: v1.10.0
• Kivy installation method: package
• Cython: 0.25.2
• Packages for SDL2: libsdl2-2.0-0 libsdl2-image-2.0-0 libsdl2-mixer-2.0-0 libsdl2-ttf-2.0-0

Description

I'm trying to run a simple kivy code under AVD.

Sometimes app run as expected and sometimes app failed with following error:

sdl2 - UnicodeDecodeError: 'utf-8' codec can't decode byte 0x98 in position 0: invalid start byte

Code and Logs

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import kivy
kivy.require('1.10.0')

from kivy.app import App
from kivy.uix.label import Label

class MyApp(App):

    def build(self):
        return Label(text='Hello world')

if __name__ == '__main__':
    MyApp().run()

Logs when it fails:

I/python  ( 2595): [CRITICAL] [Window      ] Unable to find any valuable Window provider.
I/python  ( 2595): sdl2 - UnicodeDecodeError: 'utf-8' codec can't decode byte 0x98 in position 0: invalid start byte
I/python  ( 2595):   File "/data/data/org.test.alignak/files/app/crystax_python/site-packages/kivy/core/__init__.py", line 67, in core_select_lib
I/python  ( 2595):     cls = cls()
I/python  ( 2595):   File "/data/data/org.test.alignak/files/app/crystax_python/site-packages/kivy/core/window/window_sdl2.py", line 140, in __init__
I/python  ( 2595):     super(WindowSDL, self).__init__()
I/python  ( 2595):   File "/data/data/org.test.alignak/files/app/crystax_python/site-packages/kivy/core/window/__init__.py", line 899, in __init__
I/python  ( 2595):     self.create_window()
I/python  ( 2595):   File "/data/data/org.test.alignak/files/app/crystax_python/site-packages/kivy/core/window/window_sdl2.py", line 291, in create_window
I/python  ( 2595):     super(WindowSDL, self).create_window()
I/python  ( 2595):   File "/data/data/org.test.alignak/files/app/crystax_python/site-packages/kivy/core/window/__init__.py", line 1051, in create_window
I/python  ( 2595):     self.render_context = RenderContext()
I/python  ( 2595):   File "kivy/graphics/instructions.pyx", line 752, in kivy.graphics.instructions.RenderContext.__init__ (kivy/graphics/instructions.c:12419)
I/python  ( 2595):   File "kivy/graphics/shader.pyx", line 184, in kivy.graphics.shader.Shader.__init__ (kivy/graphics/shader.c:5479)
I/python  ( 2595):   File "kivy/graphics/shader.pyx", line 693, in kivy.graphics.shader.Shader.vs.__set__ (kivy/graphics/shader.c:12909)
I/python  ( 2595):   File "kivy/graphics/shader.pyx", line 555, in kivy.graphics.shader.Shader.build_vertex (kivy/graphics/shader.c:10642)
I/python  ( 2595):   File "kivy/graphics/shader.pyx", line 578, in kivy.graphics.shader.Shader.link_program (kivy/graphics/shader.c:10951)
I/python  ( 2595):   File "kivy/graphics/shader.pyx", line 632, in kivy.graphics.shader.Shader.get_program_log (kivy/graphics/shader.c:11648)
I/python  ( 2595): [CRITICAL] [App         ] Unable to get a Window, abort.
I/ActivityManager(  984): Process org.test.alignak (pid 2595) has died.
I/WindowState(  984): WIN DEATH: Window{962104b0 u0 SurfaceView}
W/InputDispatcher(  984): channel '963c2e58 org.test.alignak/org.kivy.android.PythonActivity (server)' ~ Consumer closed input channel or an error occurred.  events=0x9
W/ActivityManager(  984): Force removing ActivityRecord{9637e278 u0 org.test.alignak/org.kivy.android.PythonActivity t8}: app died, no saved state
E/InputDispatcher(  984): channel '963c2e58 org.test.alignak/org.kivy.android.PythonActivity (server)' ~ Channel is unrecoverably broken and will be disposed!
D/Zygote  (  655): Process 2595 exited cleanly (1)
W/InputDispatcher(  984): Attempted to unregister already unregistered input channel '963c2e58 org.test.alignak/org.kivy.android.PythonActivity (server)'
I/WindowState(  984): WIN DEATH: Window{963c2e58 u0 org.test.alignak/org.kivy.android.PythonActivity}
D/EGL_emulation( 1160): eglMakeCurrent: 0xb8cddda0: ver 2 0
D/EGL_emulation( 1160): eglMakeCurrent: 0xb8cddda0: ver 2 0
E/EGL_emulation( 1160): tid 1160: eglSurfaceAttrib(1199): error 0x3009 (EGL_BAD_MATCH)
W/HardwareRenderer( 1160): Backbuffer cannot be preserved
W/InputMethodManagerService(  984): Got RemoteException sending setActive(false) notification to pid 2595 uid 10057
W/Binder  ( 1121): Caught a RuntimeException from the binder stub implementation.
W/Binder  ( 1121): java.lang.NullPointerException
W/Binder  ( 1121):  at android.inputmethodservice.IInputMethodWrapper.setSessionEnabled(IInputMethodWrapper.java:280)
W/Binder  ( 1121):  at com.android.internal.view.IInputMethod$Stub.onTransact(IInputMethod.java:129)
W/Binder  ( 1121):  at android.os.Binder.execTransact(Binder.java:404)
W/Binder  ( 1121):  at dalvik.system.NativeStart.run(Native Method)

Logs when it works:

I/python  ( 2714): os.environ is environ({'PYTHON_NAME': 'python', 'LD_LIBRARY_PATH': '/vendor/lib:/system/lib', 'PATH': '/sbin:/vendor/bin:/system/sbin:/system/bin:/system/xbin', 'PYTHONHOME': '/data/data/org.test.alignak/files/app', 'ANDROID_PROPERTY_WORKSPACE': '8,0', 'ANDROID_ASSETS': '/system/app', 'LOOP_MOUNTPOINT': '/mnt/obb', 'ANDROID_UNPACK': '/data/data/org.test.alignak/files/app', 'PYTHONOPTIMIZE': '2', 'ANDROID_ARGUMENT': '/data/data/org.test.alignak/files/app', 'ANDROID_BOOTLOGO': '1', 'ASEC_MOUNTPOINT': '/mnt/asec', 'ANDROID_ENTRYPOINT': 'main.pyo', 'BOOTCLASSPATH': '/system/framework/core.jar:/system/framework/conscrypt.jar:/system/framework/okhttp.jar:/system/framework/core-junit.jar:/system/framework/bouncycastle.jar:/system/framework/ext.jar:/system/framework/framework.jar:/system/framework/framework2.jar:/system/framework/telephony-common.jar:/system/framework/voip-common.jar:/system/framework/mms-common.jar:/system/framework/android.policy.jar:/system/framework/services.jar:/system/framework/apache-xml.jar:/system/framework/webviewchromium.jar', 'ANDROID_DATA': '/data', 'ANDROID_APP_PATH': '/data/data/org.test.alignak/files/app', 'ANDROID_ROOT': '/system', 'ANDROID_SOCKET_zygote': '9', 'EXTERNAL_STORAGE': '/storage/sdcard', 'ANDROID_PRIVATE': '/data/data/org.test.alignak/files', 'PYTHONPATH': '/data/data/org.test.alignak/files/app:/data/data/org.test.alignak/files/app/lib', 'ANDROID_STORAGE': '/storage'})
I/python  ( 2714): Android kivy bootstrap done. __name__ is __main__
I/python  ( 2714): AND: Ran string
I/python  ( 2714): Run user program, change dir and execute entrypoint
I/python  ( 2714): main.py
I/python  ( 2714): [INFO   ] [Logger      ] Record log in /data/data/org.test.alignak/files/app/.kivy/logs/kivy_18-05-06_1.txt
I/python  ( 2714): [INFO   ] [Kivy        ] v1.10.0
I/python  ( 2714): [INFO   ] [Python      ] v3.5.0 (default, Dec 24 2015, 05:56:40) 
I/python  ( 2714): [GCC 5.3 20151204]
I/python  ( 2714): [INFO   ] [Factory     ] 194 symbols loaded
I/python  ( 2714): [INFO   ] [Image       ] Providers: img_tex, img_dds, img_sdl2, img_gif (img_pil, img_ffpyplayer ignored)
I/python  ( 2714): [INFO   ] [Text        ] Provider: sdl2
I/python  ( 2714): [INFO   ] [OSC         ] using <multiprocessing> for socket
I/python  ( 2714): [INFO   ] [Window      ] Provider: sdl2
D/        ( 2714): HostConnection::get() New Host Connection established 0xb8e6eb20, tid 2728
W/        ( 2714): Unrecognized GLES max version string in extensions: ANDROID_EMU_CHECKSUM_HELPER_v1 
D/EGL_emulation( 2714): eglCreateContext: 0xb8e6f2a8: maj 2 min 0 rcv 2
D/EGL_emulation( 2714): eglMakeCurrent: 0xb8e6f2a8: ver 2 0
I/python  ( 2714): [INFO   ] [GL          ] Using the "OpenGL ES 2" graphics system
I/python  ( 2714): [INFO   ] [GL          ] Backend used <gl>
I/python  ( 2714): [INFO   ] [GL          ] OpenGL version <b'OpenGL ES 2.0 (4.6.0 NVIDIA 390.48)'>
I/python  ( 2714): [INFO   ] [GL          ] OpenGL vendor <b'Google (NVIDIA Corporation)'>
I/python  ( 2714): [INFO   ] [GL          ] OpenGL renderer <b'Android Emulator OpenGL ES Translator (GeForce GTX 760M/PCIe/SSE2)'>
I/python  ( 2714): [INFO   ] [GL          ] OpenGL parsed version: 2, 0
I/python  ( 2714): [INFO   ] [GL          ] Texture max size <16384>
I/python  ( 2714): [INFO   ] [GL          ] Texture max units <32>
I/python  ( 2714): [INFO   ] [Shader      ] fragment shader: <b'\xb0\x16j@%\x7f'>
I/python  ( 2714): [INFO   ] [Shader      ] vertex shader: <b'\x10\x80n@%\x7f'>
I/python  ( 2714): [INFO   ] [Window      ] auto add sdl2 input provider
I/python  ( 2714): [INFO   ] [Window      ] virtual keyboard not allowed, single mode, not docked
I/python  ( 2714): [WARNING] [Base        ] Unknown <android> provider
I/python  ( 2714): [INFO   ] [Base        ] Start application main loop
I/python  ( 2714): [INFO   ] [GL          ] NPOT texture support is available
I/python  ( 2714): [ERROR  ] [Base        ] Failed to import "android" module. Could not remove android presplash.
D/dalvikvm(  984): GC_CONCURRENT freed 1096K, 15% free 8330K/9740K, paused 15ms+8ms, total 86ms
D/EGL_emulation( 1035): eglMakeCurrent: 0xb8cdc0c0: ver 2 0

What I've already tried:

• clean android target and rebuilt
• uninstall Kivy and reinstall

This issue was moved by tshirtman from kivy/kivy/issues/5731.

[ghost]

@dolang commented on May 6, 2018, 9:38 AM UTC:

Looks like a driver problem.

sdl2 - UnicodeDecodeError: 'utf-8' codec can't decode byte 0x98 in position 0: invalid start byte

The byte 0x98 is a tilde in Win-1252 (cp1252) encoding but not valid utf-8. The following uses a fallback which should fix the problem:

diff --git a/kivy/graphics/shader.pyx b/kivy/graphics/shader.pyx
index 53286893..58b3227c 100644
--- a/kivy/graphics/shader.pyx
+++ b/kivy/graphics/shader.pyx
@@ -630,8 +630,13 @@ cdef class Shader:
         # others drivers doesn't include a \0 (which is great.)
         if length == 0:
             return ""
-        cdef bytes ret = msg[:length]
-        return ret.split(b'\0')[0].decode('utf-8')
+        cdef bytes ret = msg[:length].split(b'\0')[0]
+        try:
+            return ret.decode('utf-8')
+        except UnicodeDecodeError:
+            # fall back to latin1 in case the message isn't valid utf-8
+            return ret.decode('latin1')

     cdef void process_message(self, str ctype, message):
         message = message.strip()

[inclement]

Closing as stale, I haven't seen this as an issue any time recently.