beeware / briefcase-android-gradle-template

A template for generating Android Gradle projects with Briefcase
MIT License
18 stars 19 forks source link

Add Python version coverage to template CI. #55

Closed freakboy3742 closed 1 year ago

freakboy3742 commented 1 year ago

Adds python versions to the test matrix for template builds.

PR Checklist:

freakboy3742 commented 1 year ago

@mhsmith The test failures for 3.11 are to be expected; once you've published 3.11 support to Maven Central, you should be to able to re-start the failed jobs and get a clean board for merge.

mhsmith commented 1 year ago

I haven't published 3.11.0 to Maven Central yet, because there's something strange about the file sizes which didn't happen with the 3.11 RC:

target-3.11.0-2-arm64-v8a.zip
   264368  2022-10-31 16:48   lib-dynload/arm64-v8a/_codecs_jp.so
   298808  2022-10-31 16:48   lib-dynload/arm64-v8a/_decimal.so
   604912  2022-10-31 16:48   jniLibs/arm64-v8a/libssl_chaquopy.so
   613416  2022-09-17 18:34   include/sqlite3.h
  1011840  2022-10-31 16:48   jniLibs/arm64-v8a/libsqlite3_chaquopy.so
  1129192  2022-10-31 16:48   lib-dynload/arm64-v8a/unicodedata.so
  3334336  2022-10-31 16:48   jniLibs/arm64-v8a/libcrypto_chaquopy.so
  5035872  2022-10-31 16:48   jniLibs/arm64-v8a/libpython3.11.so
  5076824  2022-10-31 16:48   lib-dynload/arm64-v8a/_pickle.so
 22692548                     395 files
target-3.11.0-2-armeabi-v7a.zip
   222916  2022-10-31 16:48   lib-dynload/armeabi-v7a/_codecs_jp.so
   228668  2022-09-17 19:57   include/openssl/obj_mac.h
   237040  2022-10-31 16:48   lib-dynload/armeabi-v7a/_decimal.so
   457136  2022-10-31 16:48   jniLibs/armeabi-v7a/libssl_chaquopy.so
   613416  2022-09-17 18:34   include/sqlite3.h
   733084  2022-10-31 16:48   jniLibs/armeabi-v7a/libsqlite3_chaquopy.so
  1116400  2022-10-31 16:48   lib-dynload/armeabi-v7a/unicodedata.so
  2383776  2022-10-31 16:48   jniLibs/armeabi-v7a/libcrypto_chaquopy.so
  3397180  2022-10-31 16:48   jniLibs/armeabi-v7a/libpython3.11.so
 14086394                     395 files
target-3.11.0-2-x86_64.zip
   249176  2022-10-31 16:48   lib-dynload/x86_64/pyexpat.so
   268464  2022-10-31 16:48   lib-dynload/x86_64/_codecs_jp.so
   335672  2022-10-31 16:48   lib-dynload/x86_64/_decimal.so
   613416  2022-09-17 18:36   include/sqlite3.h
   711408  2022-10-31 16:48   jniLibs/x86_64/libssl_chaquopy.so
  1129192  2022-10-31 16:48   lib-dynload/x86_64/unicodedata.so
  1134720  2022-10-31 16:48   jniLibs/x86_64/libsqlite3_chaquopy.so
  3956928  2022-10-31 16:48   jniLibs/x86_64/libcrypto_chaquopy.so
  5449560  2022-10-31 16:48   jniLibs/x86_64/libpython3.11.so
 19205902                     395 files
target-3.11.0-2-x86.zip
   228668  2022-09-17 19:59   include/openssl/obj_mac.h
   294152  2022-10-31 16:48   lib-dynload/x86/_decimal.so
   600228  2022-10-31 16:48   jniLibs/x86/libssl_chaquopy.so
   613416  2022-09-17 18:35   include/sqlite3.h
  1007196  2022-10-31 16:48   jniLibs/x86/libsqlite3_chaquopy.so
  1120464  2022-10-31 16:48   lib-dynload/x86/unicodedata.so
  3210908  2022-10-31 16:48   jniLibs/x86/libcrypto_chaquopy.so
  4105592  2022-10-31 16:48   lib-dynload/x86/_queue.so
  4146460  2022-10-31 16:48   jniLibs/x86/libpython3.11.so
 20495171                     395 files

arm64-v8a/_pickle.so and x86/_queue.so have both massively increased in size. In fact, they're both about the same size as libpython, which suggests they've somehow ended up statically linked against it.

I'll look into this tomorrow.

mhsmith commented 1 year ago

It looks like https://github.com/python/cpython/pull/93977 introduced a bug that when cross-compiling, the Makefile no longer requires libpython to be built before it calls setup.py to build the stdlib binary modules. But exactly what happens is non-deterministic because of the parallel build. setup.py might fail to build all modules, or only some of them, or it might link some against libpython.a and some against libpython.so.

The reason I didn't notice this before is that after make we call make install, which calls setup.py again and gives it a second chance to build any modules that failed. But of course, if any modules were already linked against libpython.a, it won't build them again.

@freakboy3742: You might want to check whether this affects iOS as well. I worked around it with this patch.

freakboy3742 commented 1 year ago

@freakboy3742: You might want to check whether this affects iOS as well. I worked around it with this patch.

@mhsmith That's for the heads up; that's a nasty one. Looks like the Apple builds aren't affected though - the Apple Support package Makefile don't enable the parallel build options in the CPython makefile, so the core library is always built before the stdlib is compiled.