search

kivy / python-for-android

Turn your Python application into an Android APK
https://python-for-android.readthedocs.io
MIT License
8.11k stars 1.81k forks source link

Fix building ssl for python3 #1217

Closed agilewalker closed 5 years ago

agilewalker commented 6 years ago

Pull request https://github.com/kivy/python-for-android/pull/1195 commits squashed and rebased onto master.

ghost commented 5 years ago

@opacam use --force-redownload-p4a to disable caching of p4a with p4aspaces (it's not the default so the environment can launch offline if people really want to)

homdx commented 5 years ago

@opacam use --force-redownload-p4a to disable caching of p4a with p4aspaces (it's not the default so the environment can launch offline if people really want to)

@JonasT i seem to promote your p4a-build-spaces

homdx commented 5 years ago

@JonasT How increase build speed for p4a ? like $make -j4 ? Thanks)

opacam commented 5 years ago

I don't think you can speed up the build, this is a todo's thing, as described in issue #1558 by @KeyWeeUsr

opacam commented 5 years ago

Ei @JonasT, the branch of the fix did not contained the INSTSONAME fix that we discussed soo much....jajaja...I think that is the problem... :wink:

homdx commented 5 years ago

@opacam build and run Python2 logcat:

V/pythonutil(22541): Checking pattern libcrypto.*\.so against libSDL2_image.so
V/pythonutil(22541): Loading library: SDL2
V/pythonutil(22541): Loading library: SDL2_image
V/pythonutil(22541): Loading library: SDL2_mixer
V/pythonutil(22541): Loading library: SDL2_ttf
V/pythonutil(22541): Loading library: ssl1.1
V/pythonutil(22541): Loading library: crypto1.1
V/pythonutil(22541): Loading library: python2.7
V/pythonutil(22541): Loading library: python3.5m
V/pythonutil(22541): Library loading error: dalvik.system.PathClassLoader[DexPathList[[zip file "/data/app/com.example.test-1/base.apk"],nativeLibraryDirectories=[/data/app/com.example.test-1/lib/arm, /vendor/lib, /system/lib]]] couldn't find "libpython3.5m.so"
V/pythonutil(22541): Loading library: python3.6m
V/pythonutil(22541): Library loading error: dalvik.system.PathClassLoader[DexPathList[[zip file "/data/app/com.example.test-1/base.apk"],nativeLibraryDirectories=[/data/app/com.example.test-1/lib/arm, /vendor/lib, /system/lib]]] couldn't find "libpython3.6m.so"
V/pythonutil(22541): Loading library: python3.7m
V/pythonutil(22541): Library loading error: dalvik.system.PathClassLoader[DexPathList[[zip file "/data/app/com.example.test-1/base.apk"],nativeLibraryDirectories=[/data/app/com.example.test-1/lib/arm, /vendor/lib, /system/lib]]] couldn't find "libpython3.7m.so"
V/pythonutil(22541): Loading library: main
E/art     (22541): dlopen("/data/data/com.example.test/files/app/lib/python2.7/lib-dynload/_io.so", RTLD_LAZY) failed: dlopen failed: library "/data/data/com.example.test/files/app/lib/python2.7/lib-dynload/_io.so" not found
V/pythonutil(22541): Failed to load _io.so or unicodedata.so...but that's okay.
E/art     (22541): dlopen("/data/data/com.example.test/files/app/lib/python2.7/lib-dynload/_ctypes.so", RTLD_LAZY) failed: dlopen failed: library "/data/data/com.example.test/files/app/lib/python2.7/lib-dynload/_ctypes.so" not found
V/pythonutil(22541): Unsatisfied linker when loading ctypes
V/pythonutil(22541): Loaded everything!
I/python  (22541): Initializing Python for Android
I/python  (22541): Changing directory to the one provided by ANDROID_ARGUMENT
I/python  (22541): /data/data/com.example.test/files/app
I/python  (22541): Preparing to initialize python
I/python  (22541): _python_bundle dir exists
I/python  (22541): calculated paths to be...
I/python  (22541): /data/data/com.example.test/files/app/_python_bundle/stdlib.zip:/data/data/com.example.test/files/app/_python_bundle/modules
I/python  (22541): set wchar paths...
I/python  (22541): Initialized python
I/python  (22541): AND: Init threads
I/python  (22541): testing python print redirection
I/python  (22541): ('Android path', ['.', '', '/data/data/com.example.test/files/app/_python_bundle/stdlib.zip', '/data/data/com.example.test/files/app/_python_bundle/modules', '/data/data/com.example.test/files/app/_python_bundle/site-packages'])
I/python  (22541): ('os.environ is', {'ANDROID_APP_PATH': '/data/data/com.example.test/files/app', 'EXTERNAL_STORAGE': '/storage/emulated/legacy', 'LOOP_MOUNTPOINT': '/mnt/obb', 'SYSTEMSERVERCLASSPATH': '/system/framework/services.jar:/system/framework/ethernet-service.jar:/system/framework/wifi-service.jar', 'BOOTCLASSPATH': '/system/framework/core-libart.jar:/system/framework/conscrypt.jar:/system/framework/okhttp.jar:/system/framework/core-junit.jar:/system/framework/bouncycastle.jar:/system/framework/ext.jar:/system/framework/framework.jar:/system/framework/telephony-common.jar:/system/framework/voip-common.jar:/system/framework/ims-common.jar:/system/framework/mms-common.jar:/system/framework/android.policy.jar:/system/framework/apache-xml.jar:/system/framework/mediatek-common.jar:/system/framework/mediatek-framework.jar:/system/framework/mediatek-telephony-common.jar:/system/app/miui/miui.apk:/system/app/miuisystem/miuisystem.apk', 'ANDROID_PROPERTY_WORKSPACE': '8,0', 'PATH': '/sbin:/vendor/bin:/system/sbin:/system/bin:/system/xbin', 'ANDROID_STORAGE': '/storage', 'ANDROID_BOOTLOGO': '1', 'EMULATED_STORAGE_TARGET': '/storage/emulated', 'ANDROID_ASSETS': '/system/app', 'ANDROID_SOCKET_zygote_secondary': '9', 'PYTHONPATH': '/data/data/com.example.test/files/app:/data/data/com.example.test/files/app/lib', 'PYTHONOPTIMIZE': '2', 'ANDROID_PRIVATE': '/data/data/com.example.test/files', 'MC_AUTH_TOKEN_PATH': '/efs', 'USBOTG_STORAGE': '/storage/usbotg', 'ANDROID_ENTRYPOINT': 'main.pyo', 'SECONDARY_STORAGE': '/storage/sdcard1', 'ANDROID_UNPACK': '/data/data/com.example.test/files/app', 'ANDROID_DATA': '/data', 'EMULATED_STORAGE_SOURCE': '/mnt/shell/emulated', 'PYTHON_NAME': 'python', 'LD_PRELOAD': 'libsigchain.so:libudf.so', 'P4A_BOOTSTRAP': 'SDL2', 'ANDROID_ROOT': '/system', 'PYTHONHOME': '/data/data/com.example.test/files/app', 'ASEC_MOUNTPOINT': '/mnt/asec', 'ANDROID_ARGUMENT': '/data/data/com.example.test/files/app'})
I/python  (22541): ('Android kivy bootstrap done. __name__ is', '__main__')
I/python  (22541): ['/data/data/com.example.test/files/app/lib/python2.7/site-packages', '/data/data/com.example.test/files/app/lib/site-python']
I/python  (22541): AND: Ran string
I/python  (22541): Run user program, change dir and execute entrypoint
I/python  (22541): main.py
I/python  (22541): [WARNING] [Config      ] Older configuration version detected (0 instead of 20)
I/python  (22541): [WARNING] [Config      ] Upgrading configuration in progress.
I/python  (22541): [INFO   ] [Logger      ] Record log in /data/data/com.example.test/files/app/.kivy/logs/kivy_19-01-12_0.txt
I/python  (22541): [INFO   ] [Kivy        ] v1.10.1
I/python  (22541): [INFO   ] [Python      ] v2.7.15 (default, Jan 11 2019, 21:35:32) 
I/python  (22541): [GCC 4.2.1 Compatible Android (4691093 based on r316199) Clang 6.0.2 (https://a
I/python  (22541): [INFO   ] [Factory     ] 194 symbols loaded
I/python  (22541): [INFO   ] [Image       ] Providers: img_tex, img_dds, img_sdl2, img_gif (img_pil, img_ffpyplayer ignored)
I/python  (22541): [INFO   ] [Window      ] Provider: sdl2
I/python  (22541): [INFO   ] [GL          ] Using the "OpenGL ES 2" graphics system
I/python  (22541): [INFO   ] [GL          ] Backend used <gl>
I/python  (22541): [INFO   ] [GL          ] OpenGL version <OpenGL ES 3.1 build 1.4@3300288>
I/python  (22541): [INFO   ] [GL          ] OpenGL vendor <Imagination Technologies>
I/python  (22541): [INFO   ] [GL          ] OpenGL renderer <PowerVR Rogue G6200>
I/python  (22541): [INFO   ] [GL          ] OpenGL parsed version: 3, 1
I/python  (22541): [INFO   ] [GL          ] Texture max size <8192>
I/python  (22541): [INFO   ] [GL          ] Texture max units <16>
I/python  (22541): [INFO   ] [Window      ] auto add sdl2 input provider
I/python  (22541): [INFO   ] [Window      ] virtual keyboard not allowed, single mode, not docked
I/python  (22541): [WARNING] [Base        ] Unknown <android> provider
I/python  (22541): [INFO   ] [Base        ] Start application main loop
I/python  (22541): [INFO   ] [GL          ] NPOT texture support is available
I/python  (22541): [INFO   ] [Text        ] Provider: sdl2
I/python  (22541): [WARNING] [Accordion   ] not enough space for displaying all children
I/python  (22541): [WARNING] [Accordion   ] need 264px, got 0px
I/python  (22541): [WARNING] [Accordion   ] layout aborted.
I/python  (22541): [INFO   ] [Loader      ] using a thread pool of 2 workers
I/python  (22541): [ERROR  ] [Loader      ] Failed to load image <https://paste.opensuse.org/images/97979538.jpg>
I/python  (22541): Traceback (most recent call last):
I/python  (22541):   File "/data/data/com.example.test/files/app/_python_bundle/site-packages/kivy/loader.py", line 317, in _load_urllib
I/python  (22541):     fd = urllib_request.urlopen(filename)
I/python  (22541):   File "/data/data/com.example.test/files/app/_python_bundle/stdlib.zip/urllib2.py", line 154, in urlopen
I/python  (22541):     return opener.open(url, data, timeout)
I/python  (22541):   File "/data/data/com.example.test/files/app/_python_bundle/stdlib.zip/urllib2.py", line 429, in open
I/python  (22541):     response = self._open(req, data)
I/python  (22541):   File "/data/data/com.example.test/files/app/_python_bundle/stdlib.zip/urllib2.py", line 447, in _open
I/python  (22541):     '_open', req)
I/python  (22541):   File "/data/data/com.example.test/files/app/_python_bundle/stdlib.zip/urllib2.py", line 407, in _call_chain
I/python  (22541):     result = func(*args)
I/python  (22541):   File "/data/data/com.example.test/files/app/_python_bundle/stdlib.zip/urllib2.py", line 1241, in https_open
I/python  (22541):     context=self._context)
I/python  (22541):   File "/data/data/com.example.test/files/app/_python_bundle/stdlib.zip/urllib2.py", line 1198, in do_open
I/python  (22541):     raise URLError(err)
I/python  (22541): URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:726)>
I/python  (22541): [ERROR  ] [Loader      ] Failed to load image <https://kushaldas.in/images/pyconin15-3.jpg>
I/python  (22541): Traceback (most recent call last):
I/python  (22541):   File "/data/data/com.example.test/files/app/_python_bundle/site-packages/kivy/loader.py", line 317, in _load_urllib
I/python  (22541):     fd = urllib_request.urlopen(filename)
I/python  (22541):   File "/data/data/com.example.test/files/app/_python_bundle/stdlib.zip/urllib2.py", line 154, in urlopen
I/python  (22541):     return opener.open(url, data, timeout)
I/python  (22541):   File "/data/data/com.example.test/files/app/_python_bundle/stdlib.zip/urllib2.py", line 429, in open
I/python  (22541):     response = self._open(req, data)
I/python  (22541):   File "/data/data/com.example.test/files/app/_python_bundle/stdlib.zip/urllib2.py", line 447, in _open
I/python  (22541):     '_open', req)
I/python  (22541):   File "/data/data/com.example.test/files/app/_python_bundle/stdlib.zip/urllib2.py", line 407, in _call_chain
I/python  (22541):     result = func(*args)
I/python  (22541):   File "/data/data/com.example.test/files/app/_python_bundle/stdlib.zip/urllib2.py", line 1241, in https_open
I/python  (22541):     context=self._context)
I/python  (22541):   File "/data/data/com.example.test/files/app/_python_bundle/stdlib.zip/urllib2.py", line 1198, in do_open
I/python  (22541):     raise URLError(err)
I/python  (22541): URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:726)>
homdx commented 5 years ago

Try it now please @homdx, it should work

Note: sorry, I didn't consider the @JonasT's application p4aspaces, my master branch was outdated...I only considered the merge conflicts...

@opacam First you merge and this pull request https://github.com/kivy/python-for-android/pull/1217 are missing you previous pull request? https://github.com/kivy/python-for-android/pull/1568

opacam commented 5 years ago

yes

homdx commented 5 years ago

From my log (adb): ssl error: URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:726)> for url https://paste.opensuse.org/images/97979538.jpg I check in browser: (CN) Let's Encrypt Authority X3 (O) Let's Encrypt

Maybe request module don't know about let's encrypt certs??

opacam commented 5 years ago

About ssl certificate... that is another problem...I monkey patch my applications like this:

# Monkey patch to ssl certificate verification error
try:
    import ssl
    from functools import wraps

    print(('APPLYING MONKEY PATCH TO FORCE SSL '
          'PROTOCOL V1 [SSL VERSION: {}]'.format(
        ssl.OPENSSL_VERSION)))

    def sslwrap(func):
        @wraps(func)
        def bar(*args, **kw):
            kw['ssl_version'] = ssl.PROTOCOL_TLSv1
            return func(*args, **kw)
        return bar

    # This line below is to avoid error (detected in python-2.7.12 in linux platform):
    # URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)>
    # NOTE: This error causes that the tvshow's poster not to be shown/download when trying to load with kivy loader
    ssl._create_default_https_context = ssl._create_unverified_context

    ssl.wrap_socket = sslwrap(ssl.wrap_socket)
except Exception as e:
    print(('ERROR ON MONKEY PATCH SSL PROTOCOL V1: {}'.format(e)))

This has been working for me...probably it works for you...

homdx commented 5 years ago

@opacam and where to add this code? https://github.com/homdx/pydelhi_mobile

opacam commented 5 years ago

This should be run before any call to ssl...in you app source code...I would put somewhere before kivy loads your app (you can remove some comments I put in there and adapt to your code)

Also I would recommend you to see the following link, so you know what are you doing: https://en.wikipedia.org/wiki/Monkey_patch

:wink:

homdx commented 5 years ago

@opacam funny patch name Thanks. It's works!

I/python  (28202):   File "/data/data/com.example.test/files/app/_python_bundle/site-packages/kivy/loader.py", line 317, in _load_urllib
I/python  (28202):     fd = urllib_request.urlopen(filename)
I/python  (28202):   File "/data/data/com.example.test/files/app/_python_bundle/stdlib.zip/urllib2.py", line 154, in urlopen
I/python  (28202):     return opener.open(url, data, timeout)
I/python  (28202):   File "/data/data/com.example.test/files/app/_python_bundle/stdlib.zip/urllib2.py", line 435, in open
I/python  (28202):     response = meth(req, response)
I/python  (28202):   File "/data/data/com.example.test/files/app/_python_bundle/stdlib.zip/urllib2.py", line 548, in http_response
I/python  (28202):     'http', request, response, code, msg, hdrs)
I/python  (28202):   File "/data/data/com.example.test/files/app/_python_bundle/stdlib.zip/urllib2.py", line 473, in error
I/python  (28202):     return self._call_chain(*args)
I/python  (28202):   File "/data/data/com.example.test/files/app/_python_bundle/stdlib.zip/urllib2.py", line 407, in _call_chain
I/python  (28202):     result = func(*args)
I/python  (28202):   File "/data/data/com.example.test/files/app/_python_bundle/stdlib.zip/urllib2.py", line 556, in http_error_default
I/python  (28202):     raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
I/python  (28202): HTTPError: HTTP Error 404: Not Found
ghost commented 5 years ago

I'm pretty sure TLS below 1.1 is considered somewhat insecure, so please note that this might not be such a good idea if you want it to be properly encrypted.

Did you try shipping your own cert storage and setting it? That might also work (it worked for me)

homdx commented 5 years ago

I'm pretty sure TLS below 1.1 is considered somewhat insecure, so please note that this might not be such a good idea if you want it to be properly encrypted.

Did you try shipping your own cert storage and setting it? That might also work (it worked for me)

Tell me please. how is it done

ghost commented 5 years ago

@homdx check https://docs.python.org/3/library/ssl.html for load_verify_locations

@AndreMiras maybe it would make sense to close this pull request as well?

@agilewalker thanks so much for your work, it was very helpful as intermediate solution but we got openssl/TLS workiing on p4a master right now!

homdx commented 5 years ago

Notice: Flask with kivy in one app also are working!)) under ANDROIDAP=28 ANDROIDNDKVER=r17c NDKAPI=21 and Python3 Tkanks @opacam @JonasT @AndreMiras @agilewalker