ImportError for fitz / PyMuPDF ImportError: dlopen failed: _fitz.so is 64-bit instead of 32-bit

[robertus83]

Versions

• Python: 3.8.10
• OS: 10 [Version 10.0.19043.1706] mit WSL 2 (Ubuntu 20.04 LTS (GNU/Linux 5.10.16.3-microsoft-standard-WSL2 x86_64))
• Buildozer: 1.3.0

Description

I use the fitz / PyMuPDF Library. After deploying on any android devoice I'll get the error:

ImportError: dlopen failed: "/data/data/org.test.myapp/files/app/_python_bundle/site-packages/fitz/_fitz.so" is 64-bit instead of 32-bit

buildozer.spec

Spec file:

[app]

# (str) Title of your application
title = My Application

# (str) Package name
package.name = myapp

# (str) Package domain (needed for android/ios packaging)
package.domain = org.test

# (str) Source code where the main.py live
source.dir = .

# (list) Source files to include (let empty to include all the files)
source.include_exts = py,png,jpg,kv,atlas

# (list) List of inclusions using pattern matching
#source.include_patterns = assets/*,images/*.png

# (list) Source files to exclude (let empty to not exclude anything)
#source.exclude_exts = spec

# (list) List of directory to exclude (let empty to not exclude anything)
#source.exclude_dirs = tests, bin

# (list) List of exclusions using pattern matching
#source.exclude_patterns = license,images/*/*.jpg

# (str) Application versioning (method 1)
version = 0.1

# (str) Application versioning (method 2)
# version.regex = __version__ = ['"](.*)['"]
# version.filename = %(source.dir)s/main.py

# (list) Application requirements
# comma separated e.g. requirements = sqlite3,kivy
requirements = python3,kivy, PIL, fitz

# (str) Custom source folders for requirements
# Sets custom source for any requirements with recipes
# requirements.source.kivy = ../../kivy

# (list) Garden requirements
#garden_requirements =

# (str) Presplash of the application
#presplash.filename = %(source.dir)s/data/presplash.png

# (str) Icon of the application
#icon.filename = %(source.dir)s/data/icon.png

# (str) Supported orientation (one of landscape, sensorLandscape, portrait or all)
orientation = all

# (list) List of service to declare
#services = NAME:ENTRYPOINT_TO_PY,NAME2:ENTRYPOINT2_TO_PY

#
# OSX Specific
#

#
# author = © Copyright Info

# change the major version of python used by the app
osx.python_version = 3

# Kivy version to use
osx.kivy_version = 1.9.1

#
# Android specific
#

# (bool) Indicate if the application should be fullscreen or not
fullscreen = 0

# (string) Presplash background color (for new android toolchain)
# Supported formats are: #RRGGBB #AARRGGBB or one of the following names:
# red, blue, green, black, white, gray, cyan, magenta, yellow, lightgray,
# darkgray, grey, lightgrey, darkgrey, aqua, fuchsia, lime, maroon, navy,
# olive, purple, silver, teal.
#android.presplash_color = #FFFFFF

# (list) Permissions
#android.permissions = INTERNET

# (int) Target Android API, should be as high as possible.
#android.api = 27

# (int) Minimum API your APK will support.
#android.minapi = 21

# (int) Android SDK version to use
#android.sdk = 20

# (str) Android NDK version to use
#android.ndk = 19b

# (int) Android NDK API to use. This is the minimum API your app will support, it should usually match android.minapi.
#android.ndk_api = 21

# (bool) Use --private data storage (True) or --dir public storage (False)
#android.private_storage = True

# (str) Android NDK directory (if empty, it will be automatically downloaded.)
#android.ndk_path =

# (str) Android SDK directory (if empty, it will be automatically downloaded.)
#android.sdk_path =

# (str) ANT directory (if empty, it will be automatically downloaded.)
#android.ant_path =

# (bool) If True, then skip trying to update the Android sdk
# This can be useful to avoid excess Internet downloads or save time
# when an update is due and you just want to test/build your package
# android.skip_update = False

# (bool) If True, then automatically accept SDK license
# agreements. This is intended for automation only. If set to False,
# the default, you will be shown the license when first running
# buildozer.
# android.accept_sdk_license = False

# (str) Android entry point, default is ok for Kivy-based app
#android.entrypoint = org.renpy.android.PythonActivity

# (str) Android app theme, default is ok for Kivy-based app
# android.apptheme = "@android:style/Theme.NoTitleBar"

# (list) Pattern to whitelist for the whole project
#android.whitelist =

# (str) Path to a custom whitelist file
#android.whitelist_src =

# (str) Path to a custom blacklist file
#android.blacklist_src =

# (list) List of Java .jar files to add to the libs so that pyjnius can access
# their classes. Don't add jars that you do not need, since extra jars can slow
# down the build process. Allows wildcards matching, for example:
# OUYA-ODK/libs/*.jar
#android.add_jars = foo.jar,bar.jar,path/to/more/*.jar

# (list) List of Java files to add to the android project (can be java or a
# directory containing the files)
#android.add_src =

# (list) Android AAR archives to add (currently works only with sdl2_gradle
# bootstrap)
#android.add_aars =

# (list) Gradle dependencies to add (currently works only with sdl2_gradle
# bootstrap)
#android.gradle_dependencies =

# (list) add java compile options
# this can for example be necessary when importing certain java libraries using the 'android.gradle_dependencies' option
# see https://developer.android.com/studio/write/java8-support for further information
# android.add_compile_options = "sourceCompatibility = 1.8", "targetCompatibility = 1.8"

# (list) Gradle repositories to add {can be necessary for some android.gradle_dependencies}
# please enclose in double quotes 
# e.g. android.gradle_repositories = "maven { url 'https://kotlin.bintray.com/ktor' }"
#android.add_gradle_repositories =

# (list) packaging options to add 
# see https://google.github.io/android-gradle-dsl/current/com.android.build.gradle.internal.dsl.PackagingOptions.html
# can be necessary to solve conflicts in gradle_dependencies
# please enclose in double quotes 
# e.g. android.add_packaging_options = "exclude 'META-INF/common.kotlin_module'", "exclude 'META-INF/*.kotlin_module'"
#android.add_gradle_repositories =

# (list) Java classes to add as activities to the manifest.
#android.add_activities = com.example.ExampleActivity

# (str) OUYA Console category. Should be one of GAME or APP
# If you leave this blank, OUYA support will not be enabled
#android.ouya.category = GAME

# (str) Filename of OUYA Console icon. It must be a 732x412 png image.
#android.ouya.icon.filename = %(source.dir)s/data/ouya_icon.png

# (str) XML file to include as an intent filters in <activity> tag
#android.manifest.intent_filters =

# (str) launchMode to set for the main activity
#android.manifest.launch_mode = standard

# (list) Android additional libraries to copy into libs/armeabi
#android.add_libs_armeabi = libs/android/*.so
#android.add_libs_armeabi_v7a = libs/android-v7/*.so
#android.add_libs_arm64_v8a = libs/android-v8/*.so
#android.add_libs_x86 = libs/android-x86/*.so
#android.add_libs_mips = libs/android-mips/*.so

# (bool) Indicate whether the screen should stay on
# Don't forget to add the WAKE_LOCK permission if you set this to True
#android.wakelock = False

# (list) Android application meta-data to set (key=value format)
#android.meta_data =

# (list) Android library project to add (will be added in the
# project.properties automatically.)
#android.library_references =

# (list) Android shared libraries which will be added to AndroidManifest.xml using <uses-library> tag
#android.uses_library =

# (str) Android logcat filters to use
#android.logcat_filters = *:S python:D

# (bool) Copy library instead of making a libpymodules.so
#android.copy_libs = 1

# (str) The Android arch to build for, choices: armeabi-v7a, arm64-v8a, x86, x86_64
android.arch = armeabi-v7a

# (int) overrides automatic versionCode computation (used in build.gradle)
# this is not the same as app version and should only be edited if you know what you're doing
# android.numeric_version = 1

#
# Python for android (p4a) specific
#

# (str) python-for-android fork to use, defaults to upstream (kivy)
#p4a.fork = kivy

# (str) python-for-android branch to use, defaults to master
#p4a.branch = master

# (str) python-for-android git clone directory (if empty, it will be automatically cloned from github)
#p4a.source_dir =

# (str) The directory in which python-for-android should look for your own build recipes (if any)
#p4a.local_recipes =

# (str) Filename to the hook for p4a
#p4a.hook =

# (str) Bootstrap to use for android builds
# p4a.bootstrap = sdl2

# (int) port number to specify an explicit --port= p4a argument (eg for bootstrap flask)
#p4a.port =

#
# iOS specific
#

# (str) Path to a custom kivy-ios folder
#ios.kivy_ios_dir = ../kivy-ios
# Alternately, specify the URL and branch of a git checkout:
ios.kivy_ios_url = https://github.com/kivy/kivy-ios
ios.kivy_ios_branch = master

# Another platform dependency: ios-deploy
# Uncomment to use a custom checkout
#ios.ios_deploy_dir = ../ios_deploy
# Or specify URL and branch
ios.ios_deploy_url = https://github.com/phonegap/ios-deploy
ios.ios_deploy_branch = 1.7.0

# (str) Name of the certificate to use for signing the debug version
# Get a list of available identities: buildozer ios list_identities
#ios.codesign.debug = "iPhone Developer: <lastname> <firstname> (<hexstring>)"

# (str) Name of the certificate to use for signing the release version
#ios.codesign.release = %(ios.codesign.debug)s

[buildozer]

# (int) Log level (0 = error only, 1 = info, 2 = debug (with command output))
log_level = 2

# (int) Display warning if buildozer is run as root (0 = False, 1 = True)
warn_on_root = 1

# (str) Path to build artifact storage, absolute or relative to spec file
# build_dir = ./.buildozer

# (str) Path to build output (i.e. .apk, .ipa) storage
# bin_dir = ./bin

#    -----------------------------------------------------------------------------
#    List as sections
#
#    You can define all the "list" as [section:key].
#    Each line will be considered as a option to the list.
#    Let's take [app] / source.exclude_patterns.
#    Instead of doing:
#
#[app]
#source.exclude_patterns = license,data/audio/*.wav,data/images/original/*
#
#    This can be translated into:
#
#[app:source.exclude_patterns]
#license
#data/audio/*.wav
#data/images/original/*
#

#    -----------------------------------------------------------------------------
#    Profiles
#
#    You can extend section / key with a profile
#    For example, you want to deploy a demo version of your application without
#    HD content. You could first change the title to add "(demo)" in the name
#    and extend the excluded directories to remove the HD content.
#
#[app@demo]
#title = My Application (demo)
#
#[app:source.exclude_patterns@demo]
#images/hd/*
#
#    Then, invoke the command line with the "demo" profile:
#
#buildozer --profile demo android debug

[misl6]

Hi @robertus83 , Seems that fitz is not a plain-python package, and may need a recipe.

See: https://python-for-android.readthedocs.io/en/latest/recipes/ for further information about recipes. Feel free to join us on Discord if you need some help or guidance.

EDIT: Moved to python-for-android and marked as feature-request

[domdrag]

Not fully related, but this might help someone. You might want to check out pdfplumber and/or PyPDF2 packages. I originally wanted to use PyMuPDF for converting PDF to PNG in my Android app and stumbled upon the same issue. Then realized this could also be done with pdfplumber which is a pure python package. My guess would be the packages share a great deal of features. Also quoting from their github page :

pymupdf is substantially faster than pdfminer.six (and thus also pdfplumber) and can generate and modify PDFs, but the library requires installation of non-Python software (MuPDF). It also does not enable easy access to shape objects (rectangles, lines, etc.), and does not provide table-extraction or visual debugging tools.

[wishrohitv]

Not fully related, but this might help someone. You might want to check out pdfplumber and/or PyPDF2 packages. I originally wanted to use PyMuPDF for converting PDF to PNG in my Android app and stumbled upon the same issue. Then realized this could also be done with pdfplumber which is a pure python package. My guess would be the packages share a great deal of features. Also quoting from their github page :

pymupdf is substantially faster than pdfminer.six (and thus also pdfplumber) and can generate and modify PDFs, but the library requires installation of non-Python software (MuPDF). It also does not enable easy access to shape objects (rectangles, lines, etc.), and does not provide table-extraction or visual debugging tools.

have you succeeded to convert pdf to png in android phone?

[domdrag]

@wish-rohitv Yes I have. In the code, I used the pdfplumber's to_image function which converts a pdf page to an image. For Android deployment, you need to do some extra steps to make it work. The to_image function uses pypdfium2 which is a platform-specific library so you need to include the binaries targeting Android OS into your project manually. The binaries can be found here. Make sure to update the buildozer.spec file so the buildozer is aware of the binaries you include.

[wishrohitv]

@wish-rohitv Yes I have. In the code, I used the pdfplumber's to_image function which converts a pdf page to an image. For Android deployment, you need to do some extra steps to make it work. The page_to_image function uses pypdfium2 which is a platform-specific library so you need to include the binaries targeting Android OS into your project manually. The binaries can be found here. Make sure to update the buildozer.spec file so the buildozer is aware of the binaries you include.

thank for reply sir, can you explain steps, how to add pydfium2 in code and which one i should pick

[domdrag]

@wish-rohitv You don't need to add pypdfium2 in your code, it is already part of the pdfplumber's library. You just need to use to_image function to convert a pdf page to an image as shown in this example. For your second question, it depends on what architecture target is set in your buildozer.spec file. By default, the relevant part of the buildozer.spec file should look like this

...
android.archs = arm64-v8a, armeabi-v7a
...

If so, for arm64-v8a CPU you should download pdfium-v8-android-arm64.tgz and extract libpdfium.so to [your-project]/libs/android-v8/ directory. After that, uncomment android.add_libs_arm64_v8a = libs/android-v8/*.so in your buildozer.spec file. For armeabi-v7a CPU, download pdfium-android-arm.tgz, extract the .so file to [your-project]/libs/android/ [your-project]/libs/android-v7/ directory and uncomment android.add_libs_armeabi = libs/android/*.so *android.add_libs_armeabi_v7a = libs/android-v7/.so** in your buildozer.spec.

[wishrohitv]

@wish-rohitv You don't need to add pypdfium2 in your code, it is already part of the pdfplumber's library. You just need to use to_image function to convert a pdf page to an image as shown in this example. For your second question, it depends on what architecture target is set in your buildozer.spec file. By default, the relevant part of the buildozer.spec file should look like this

...
android.archs = arm64-v8a, armeabi-v7a
...

If so, for arm64-v8a CPU you should download pdfium-v8-android-arm64.tgz and extract libpdfium.so to [your-project]/libs/android-v8/ directory. After that, uncomment android.add_libs_arm64_v8a = libs/android-v8/*.so in your buildozer.spec file. For armeabi-v7a CPU, download pdfium-android-arm.tgz, extract the .so file to [your-project]/libs/android/ directory and uncomment android.add_libs_arm64_v8a = libs/android/*.so in your buildozer.spec.

thankyou show much🥰, sorry for being late my laptop was damaged, libpdfium.so file should be in project directory and android.add_libs_arm64_v8a = libs/android-v8/*.so be uncommented , isn't it?

[domdrag]

@wish-rohitv Yes, make sure libpdfium.so is placed in the correct directory according to the buildozer.spec. If you will only uncomment the corresponding line, place the libpdfium.so in [your-project]/libs/android-v8/.

[domdrag]

@wish-rohitv In directory where your buildozer.spec file exists, create libs/android-v8/ directory and place corresponding libpdfium.so in there.

[wishrohitv]

@wish-rohitv In directory where your buildozer.spec file exists, create libs/android-v8/ directory and place corresponding libpdfium.so in there.

You mean like this - [main.py,main.kv, buildozer.spec, libs/android-v8/libpdfium.so] ?

[domdrag]

@wish-rohitv Yes.

[wishrohitv]

@wish-rohitv Yes.

Thanks for your valuable assistance😊

[domdrag]

@wish-rohitv Include pdfminer.six module in requirements in buildozer.spec.

[domdrag]

@wish-rohitv This one should be fixed by adding charset_normalizer==2.1.1 in requirements in buildozer.spec.

[wishrohitv]

@wish-rohitv This one should be fixed by adding charset_normalizer==2.1.1 in requirements in buildozer.spec.

Thanks 😊, it additionally required more modules such as `pypdfium2, cryptography, docutilts in buildozer requirements

[wishrohitv]

hey i'm getting error pdfium.so is 64-bit instead of 32-bit i have followed procedure very carefully i have used pdfium-android-arm.tgz for cpu arm 32 bit android in my project please help

[domdrag]

@wish-rohitv I would assume you placed incorrect .so file inside libs/android directory. Make sure that your 32-bit libpdfium.so is placed inside libs/android directory and that you have android.add_libs_armeabi = libs/android/*.so set in your buildozer.spec. My original comment from Sep 27 had a mistake, now it has been fixed.

[wishrohitv]

i'm getting again same error

is my steps wrong -

[domdrag]

@wish-rohitv Not sure what to tell you. Maybe you could try building apk for arm64-v8a and place the corresponding libpdfium.so in libs/android-v8/ directory? Unless you have a super old phone, it should be supported.

[wishrohitv]

i don't have super old phone, i tested on android 11, 9, 6 samsung phones 32 bit cpu can you tell me what is pdfium v8

[domdrag]

@wish-rohitv Same thing as pdfium, just for different CPU architecture.

[wishrohitv]

actually arm64-v8a doesn't install on my phones

should i put armeabi instead of armeabi-v7a?

[domdrag]

@wish-rohitv I don't think so. Maybe you need to clean your build after modifying additional libraries. Try using buildozer appclean command and try to build again your previous tries.

[wishrohitv]

actually I'm using GitHub action because every time building apk while making little changes is very pain full ok i will do buildozer app clean thanks for your time sir

[domdrag]

@wishrohitv Hi, I was getting the same issue you were facing when I deleted buildozer build files for my app and it took me a while to figure out the issue. Basically, in newer releases of pypdfium2, the package comes with the built-in libpdfium.so which is designed for x64 arch and because of that, I'd say buildozer doesn't try to find it in your own provided library inside the libs directory. To avoid that you need to include the earlier release of pypdfium2 (e.g. 4.20.0 worked for me). So instead of placing pypdfium2 in your requirements in buildozer.spec, try placing pypfdium2==4.20.0.

Additionally, I've given you false information on providing your own libraries for armeabi-v7a. So let me correct myself;

@wish-rohitv You don't need to add pypdfium2 in your code, it is already part of the pdfplumber's library. You just need to use to_image function to convert a pdf page to an image as shown in this example. For your second question, it depends on what architecture target is set in your buildozer.spec file. By default, the relevant part of the buildozer.spec file should look like this

...
android.archs = arm64-v8a, armeabi-v7a
...

If so, for arm64-v8a CPU you should download pdfium-v8-android-arm64.tgz and extract libpdfium.so to [your-project]/libs/android-v8/ directory. After that, uncomment android.add_libs_arm64_v8a = libs/android-v8/*.so in your buildozer.spec file. For armeabi-v7a CPU, download pdfium-android-arm.tgz, extract the .so file to [your-project]/libs/android/ [your-project]/libs/android-v7/ directory and uncomment android.add_libs_armeabi = libs/android/*.so *android.add_libs_armeabi_v7a = libs/android-v7/.so** in your buildozer.spec.

The erroneous comment has been fixed. Also, although I don't think is necessary, you might uncomment android.copy_libs = 1 in your buildozer.spec. Make sure to always run buildozer appclean whenever modifying versions of packages, changing architectures and providing/changing your own libraries in buildozer.spec!

Apologies for the misinformations and misguidance.

[wishrohitv]

@domdrag thanks again for your guide sir, i left the pdfplumer library when it doesn't work for me and then i tried another approach for render pdf on android phone. hope your advice will help another people in future