Error when execute APK only on device: ImportError: cannot import name _htmlparser

[joaopaulosilvasimoes]

Enviroment:

• Python: v2.7.2
• OS: Xubuntu 18.04 LTS / Android 7
• Kivy: v1.10.0
• Cython: 0.25.2

-- Issue:

I am developing an application that is working correctly when I run directly on my computer, however I run on my phone it presents a series of errors related to BS4 (BeautifulSoup).

As one of the libraries I use is "Skpy" https://pypi.org/project/SkPy, and it makes use of BeautifulSoap, so I can not substitute it directly, since it's a third-party library.

When I comment the use of the library in the application works normally, both on the PC and on the device.

I've done tests with Python 3.6 using NDK and the current Python 2.7 model, and the error is simply the same, I do not know how to proceed.

Has anyone ever experienced this?

-- Sample:

# https://skpy.t.allofti.me/
from skpy import Skype

# https://pandas.pydata.org/
import pandas as pd

class SkypeBackup:

    def __init__(self, user, password, connect=False):

        self.user     = user
        self.password = password
        self.connect  = connect

        self.sk       = Skype(self.user, self.password, self.connect)

    def get_contacts_csv(self, sep=';', encoding="utf-8"):

        self.data_frame_contacts = pd.DataFrame([vars(contact) for contact in self.sk.contacts])

        return self.data_frame_contacts.to_csv(sep=sep, encoding=encoding)

-- buildozer.spec

[app]

# (str) Title of your application
title = SkypeBackup

# (str) Package name
package.name = skypebackupapp

# (str) Package domain (needed for android/ios packaging)
package.domain = com.gmail

# (str) Source code where the main.py live
source.dir = .

# (list) Source files to include (let empty to include all the files)
source.include_exts = py,png,jpg,kv,atlas,otf

# (list) List of inclusions using pattern matching
#source.include_patterns = assets/*,images/*.png

# (list) Source files to exclude (let empty to not exclude anything)
#source.exclude_exts = spec

# (list) List of directory to exclude (let empty to not exclude anything)
#source.exclude_dirs = tests, bin

# (list) List of exclusions using pattern matching
#source.exclude_patterns = license,images/*/*.jpg

# (str) Application versioning (method 1)
version = 0.40

# (str) Application versioning (method 2)
# version.regex = __version__ = ['"](.*)['"]
# version.filename = %(source.dir)s/main.py

# (list) Application requirements
# comma separated e.g. requirements = sqlite3,kivy
requirements = pandas,kivy,beautifulsoup4,skpy

# (str) Custom source folders for requirements
# Sets custom source for any requirements with recipes
# requirements.source.kivy = ../../kivy

# (list) Garden requirements
#garden_requirements =

# (str) Presplash of the application
presplash.filename = %(source.dir)s/data/icon.png

# (str) Icon of the application
icon.filename = %(source.dir)s/data/icon.png

# (str) Supported orientation (one of landscape, portrait or all)
orientation = portrait

# (list) List of service to declare
#services = NAME:ENTRYPOINT_TO_PY,NAME2:ENTRYPOINT2_TO_PY

#
# OSX Specific
#

#
# author = © Copyright Info

# change the major version of python used by the app
osx.python_version = 3

# Kivy version to use
osx.kivy_version = 1.9.1

#
# Android specific
#

# (bool) Indicate if the application should be fullscreen or not
fullscreen = 0

# (string) Presplash background color (for new android toolchain)
# Supported formats are: #RRGGBB #AARRGGBB or one of the following names:
# red, blue, green, black, white, gray, cyan, magenta, yellow, lightgray,
# darkgray, grey, lightgrey, darkgrey, aqua, fuchsia, lime, maroon, navy,
# olive, purple, silver, teal.
android.presplash_color = #00aff0

# (list) Permissions
android.permissions = INTERNET

# (int) Android API to use
#android.api = 19

# (int) Minimum API required
#android.minapi = 9

# (int) Android SDK version to use
#android.sdk = 20

# (str) Android NDK version to use
#android.ndk = 10.3.2

# (bool) Use --private data storage (True) or --dir public storage (False)
#android.private_storage = True

# (str) Android NDK directory (if empty, it will be automatically downloaded.)
#android.ndk_path = /home/xubuntu/crystax/crystax-ndk-10.3.2/

# (str) Android SDK directory (if empty, it will be automatically downloaded.)
#android.sdk_path =

# (str) ANT directory (if empty, it will be automatically downloaded.)
#android.ant_path =

# (bool) If True, then skip trying to update the Android sdk
# This can be useful to avoid excess Internet downloads or save time
# when an update is due and you just want to test/build your package
# android.skip_update = False

# (str) Android entry point, default is ok for Kivy-based app
#android.entrypoint = org.renpy.android.PythonActivity

# (list) Pattern to whitelist for the whole project
#android.whitelist =

# (str) Path to a custom whitelist file
#android.whitelist_src =

# (str) Path to a custom blacklist file
#android.blacklist_src =

# (list) List of Java .jar files to add to the libs so that pyjnius can access
# their classes. Don't add jars that you do not need, since extra jars can slow
# down the build process. Allows wildcards matching, for example:
# OUYA-ODK/libs/*.jar
#android.add_jars = foo.jar,bar.jar,path/to/more/*.jar

# (list) List of Java files to add to the android project (can be java or a
# directory containing the files)
#android.add_src =

# (list) Android AAR archives to add (currently works only with sdl2_gradle
# bootstrap)
#android.add_aars =

# (list) Gradle dependencies to add (currently works only with sdl2_gradle
# bootstrap)
#android.gradle_dependencies =

# (list) Java classes to add as activities to the manifest.
#android.add_activites = com.example.ExampleActivity

# (str) python-for-android branch to use, defaults to stable
#p4a.branch = stable

# (str) OUYA Console category. Should be one of GAME or APP
# If you leave this blank, OUYA support will not be enabled
#android.ouya.category = GAME

# (str) Filename of OUYA Console icon. It must be a 732x412 png image.
#android.ouya.icon.filename = %(source.dir)s/data/ouya_icon.png

# (str) XML file to include as an intent filters in <activity> tag
#android.manifest.intent_filters =

# (str) launchMode to set for the main activity
#android.manifest.launch_mode = standard

# (list) Android additional libraries to copy into libs/armeabi
#android.add_libs_armeabi = libs/android/*.so
#android.add_libs_armeabi_v7a = libs/android-v7/*.so
#android.add_libs_x86 = libs/android-x86/*.so
#android.add_libs_mips = libs/android-mips/*.so

# (bool) Indicate whether the screen should stay on
# Don't forget to add the WAKE_LOCK permission if you set this to True
#android.wakelock = False

# (list) Android application meta-data to set (key=value format)
#android.meta_data =

# (list) Android library project to add (will be added in the
# project.properties automatically.)
#android.library_references =

# (str) Android logcat filters to use
#android.logcat_filters = *:S python:D

# (bool) Copy library instead of making a libpymodules.so
#android.copy_libs = 1

# (str) The Android arch to build for, choices: armeabi-v7a, arm64-v8a, x86
android.arch = armeabi-v7a

#
# Python for android (p4a) specific
#

# (str) python-for-android git clone directory (if empty, it will be automatically cloned from github)
#p4a.source_dir =

# (str) The directory in which python-for-android should look for your own build recipes (if any)
#p4a.local_recipes =

# (str) Filename to the hook for p4a
#p4a.hook =

# (str) Bootstrap to use for android builds
# p4a.bootstrap = sdl2

# (int) port number to specify an explicit --port= p4a argument (eg for bootstrap flask)
#p4a.port =

#
# iOS specific
#

# (str) Path to a custom kivy-ios folder
#ios.kivy_ios_dir = ../kivy-ios

# (str) Name of the certificate to use for signing the debug version
# Get a list of available identities: buildozer ios list_identities
#ios.codesign.debug = "iPhone Developer: <lastname> <firstname> (<hexstring>)"

# (str) Name of the certificate to use for signing the release version
#ios.codesign.release = %(ios.codesign.debug)s

[buildozer]

# (int) Log level (0 = error only, 1 = info, 2 = debug (with command output))
log_level = 2

# (int) Display warning if buildozer is run as root (0 = False, 1 = True)
warn_on_root = 1

# (str) Path to build artifact storage, absolute or relative to spec file
# build_dir = ./.buildozer

# (str) Path to build output (i.e. .apk, .ipa) storage
# bin_dir = ./bin

#    -----------------------------------------------------------------------------
#    List as sections
#
#    You can define all the "list" as [section:key].
#    Each line will be considered as a option to the list.
#    Let's take [app] / source.exclude_patterns.
#    Instead of doing:
#
#[app]
#source.exclude_patterns = license,data/audio/*.wav,data/images/original/*
#
#    This can be translated into:
#
#[app:source.exclude_patterns]
#license
#data/audio/*.wav
#data/images/original/*
#

#    -----------------------------------------------------------------------------
#    Profiles
#
#    You can extend section / key with a profile
#    For example, you want to deploy a demo version of your application without
#    HD content. You could first change the title to add "(demo)" in the name
#    and extend the excluded directories to remove the HD content.
#
#[app@demo]
#title = My Application (demo)
#
#[app:source.exclude_patterns@demo]
#images/hd/*
#
#    Then, invoke the command line with the "demo" profile:
#
#buildozer --profile demo android debug

-- Log:

12-12 15:10:08.097 17226 17300 I python  : Initialize Python for Android
12-12 15:10:08.097 17226 17300 I python  : Changing directory to the one provided by ANDROID_ARGUMENT
12-12 15:10:08.097 17226 17300 I python  : /data/user/0/com.gmail.skypebackupapp/files/app
12-12 15:10:08.097 17226 17300 I python  : Preparing to initialize python
12-12 15:10:08.097 17226 17300 I python  : crystax_python does not exist
12-12 15:10:08.114 17226 17300 I python  : Initialized python
12-12 15:10:08.114 17226 17300 I python  : AND: Init threads
12-12 15:10:08.116 17226 17300 I python  : testing python print redirection
12-12 15:10:08.118 17226 17300 I python  : Setting up python from ANDROID_PRIVATE
12-12 15:10:08.120 17226 17300 I python  : ('Android path', ['/data/user/0/com.gmail.skypebackupapp/files/app/lib/python27.zip', '/data/user/0/com.gmail.skypebackupapp/files/app/lib/python2.7/', '/data/user/0/com.gmail.skypebackupapp/files/app/lib/python2.7/lib-dynload/', '/data/user/0/com.gmail.skypebackupapp/files/app/lib/python2.7/site-packages/', '/data/user/0/com.gmail.skypebackupapp/files/app'])
12-12 15:10:08.147 17226 17300 I python  : ('os.environ is', {'DOWNLOAD_CACHE': '/data/cache', 'ANDROID_UNPACK': '/data/user/0/com.gmail.skypebackupapp/files/app', 'EXTERNAL_STORAGE': '/sdcard', 'ANDROID_ENTRYPOINT': 'main.pyo', 'ANDROID_ASSETS': '/system/app', 'PYTHONPATH': '/data/user/0/com.gmail.skypebackupapp/files/app:/data/user/0/com.gmail.skypebackupapp/files/app/lib', 'ANDROID_APP_PATH': '/data/user/0/com.gmail.skypebackupapp/files/app', 'ANDROID_ROOT': '/system', 'SYSTEMSERVERCLASSPATH': '/system/framework/services.jar:/system/framework/ethernet-service.jar:/system/framework/wifi-service.jar', 'PYTHONOPTIMIZE': '2', 'ANDROID_DATA': '/data', 'PYTHONHOME': '/data/user/0/com.gmail.skypebackupapp/files/app', 'ANDROID_ARGUMENT': '/data/user/0/com.gmail.skypebackupapp/files/app', 'BOOTCLASSPATH': '/system/framework/core-oj.jar:/system/framework/core-libart.jar:/system/framework/conscrypt.jar:/system/framework/okhttp.jar:/system/framework/core-junit.jar:/system/framework/bouncycastle.jar:/system/framework/ext.jar:/system/framework/framework.jar:/system/framework/telephony-common.jar:/system/framework/voip-common.jar:/system/framework/ims-common.jar:/system/framework/apache-xml.jar:/system/framework/org.apache.http.legacy.boot.jar:/system/framework/mediatek-common.jar:/system/framework/mediatek-framework.jar:/system/framework/mediatek-telephony-common.jar', 'ANDROID_SOCKET_zygote_secondary': '8', 'ASEC_MOUNTPOINT': '/mnt/asec', 'PATH': '/sbin:/vendor/bin:/system/sbin:/system/bin:/system/xbin', 'ANDROID_STORAGE': '/storage', 'ANDROID_BOOTLOGO': '1', 'ANDROID_PRIVATE': '/data/user/0/com.gmail.skypebackupapp/files', 'PYTHON_NAME': 'python'})
12-12 15:10:08.147 17226 17300 I python  : ('Android kivy bootstrap done. __name__ is', '__main__')
12-12 15:10:08.183 17226 17300 I python  : ['/data/user/0/com.gmail.skypebackupapp/files/app/lib/python2.7/site-packages', '/data/user/0/com.gmail.skypebackupapp/files/app/lib/site-python']
12-12 15:10:08.183 17226 17300 I python  : AND: Ran string
12-12 15:10:08.183 17226 17300 I python  : Run user program, change dir and execute entrypoint
12-12 15:10:08.296 17226 17300 I python  : [WARNING] [Config      ] Older configuration version detected (0 instead of 19)
12-12 15:10:08.296 17226 17300 I python  : [WARNING] [Config      ] Upgrading configuration in progress.
12-12 15:10:08.304 17226 17300 I python  : [INFO   ] [Logger      ] Record log in /data/user/0/com.gmail.skypebackupapp/files/app/.kivy/logs/kivy_18-12-12_0.txt
12-12 15:10:08.305 17226 17300 I python  : [INFO   ] [Kivy        ] v1.10.0
12-12 15:10:08.305 17226 17300 I python  : [INFO   ] [Python      ] v2.7.2 (default, Dec  3 2018, 17:05:44) 
12-12 15:10:08.305 17226 17300 I python  : [GCC 4.8]
12-12 15:10:08.341 17226 17300 I python  : [INFO   ] [Factory     ] 194 symbols loaded
12-12 15:10:08.467  2215  2526 E Accel   : MPE_acc_current_data -0.340973 0.264038 -0.233184,3
12-12 15:10:08.701  2215  2526 E GYRO    : MPE_gyro_current_data 0.088333 -0.175066 0.544207,3,0.000000 0.000000 0.000000,-0.088333 0.175066 -0.544207,0.000000
12-12 15:10:08.812 17226 17300 I python  : [INFO   ] [Image       ] Providers: img_tex, img_dds, img_sdl2, img_gif (img_pil, img_ffpyplayer ignored)
12-12 15:10:08.901 17226 17300 I python  :  Traceback (most recent call last):
12-12 15:10:08.901 17226 17300 I python  :    File "/home/xubuntu/Projects/skypebackup/skypebackup-python2/src/.buildozer/android/app/main.py", line 10, in <module>
12-12 15:10:08.902 17226 17300 I python  :    File "/home/xubuntu/Projects/skypebackup/skypebackup-python2/src/.buildozer/android/app/login.py", line 6, in <module>
12-12 15:10:08.902 17226 17300 I python  :    File "/home/xubuntu/Projects/skypebackup/skypebackup-python2/src/.buildozer/android/app/skypebackup.py", line 3, in <module>
12-12 15:10:08.903 17226 17300 I python  :    File "/home/xubuntu/Projects/skypebackup/skypebackup-python2/src/.buildozer/android/platform/build/dists/skypebackupapp/private/lib/python2.7/site-packages/skpy/__init__.py", line 6, in <module>
12-12 15:10:08.903 17226 17300 I python  :    File "/home/xubuntu/Projects/skypebackup/skypebackup-python2/src/.buildozer/android/platform/build/dists/skypebackupapp/private/lib/python2.7/site-packages/skpy/util.py", line 7, in <module>
12-12 15:10:08.904 17226 17300 I python  :    File "/home/xubuntu/Projects/skypebackup/skypebackup-python2/src/.buildozer/android/platform/build/dists/skypebackupapp/private/lib/python2.7/site-packages/skpy/conn.py", line 10, in <module>
12-12 15:10:08.904 17226 17300 I python  :    File "/home/xubuntu/Projects/skypebackup/skypebackup-python2/src/.buildozer/android/platform/build/dists/skypebackupapp/private/lib/python2.7/site-packages/bs4/__init__.py", line 36, in <module>
12-12 15:10:08.905 17226 17300 I python  :    File "/home/xubuntu/Projects/skypebackup/skypebackup-python2/src/.buildozer/android/platform/build/dists/skypebackupapp/private/lib/python2.7/site-packages/bs4/builder/__init__.py", line 326, in <module>
12-12 15:10:08.905 17226 17300 I python  :  ImportError: cannot import name _htmlparser
12-12 15:10:08.950 17226 17300 I python  : Python for android ended.
12-12 15:10:08.966  2215  4889 E Sensors : handleToDriver handle(0)

[intellign]

Add this before importing beautifulsoup4

import imp
import sys

class ImportBlocker(object):

    def __init__(self, *args):
        self.black_list = args

    def find_module(self, name, path=None):
        if name in self.black_list:
            return self

        return None

    def load_module(self, name):
        module = imp.new_module(name)
        module.__all__ = [] # Necessary because of how bs4 inspects the module

        return module

sys.meta_path = [ImportBlocker('bs4.builder._htmlparser')]

source