search

chaquo / chaquopy

Chaquopy: the Python SDK for Android
https://chaquo.com/chaquopy/
MIT License
850 stars 132 forks source link

Sentencepiece for Python 3.9 or later #1264

Open aryan1165 opened 1 month ago

aryan1165 commented 1 month ago

I was using briefca, specifically beeware to build an android app. When i use command to build app for android, it is throwing me this error.

Setting main module... done

[nmtbhashini] Building Android APK...
Starting a Gradle Daemon (subsequent builds will be faster)

> Configure project :app
Warning: Python version 3.11 may have fewer packages available. If you experience problems, try switching to version 3.8.

> Task :app:preBuild UP-TO-DATE
> Task :app:preDebugBuild UP-TO-DATE
> Task :app:mergeDebugNativeDebugMetadata NO-SOURCE
> Task :app:extractDebugPythonBuildPackages UP-TO-DATE

> Task :app:generateDebugPythonRequirements
Chaquopy: Installing for arm64-v8a
Looking in indexes: https://pypi.org/simple, https://chaquo.com/pypi-13.1
Collecting numpy
  Using cached https://chaquo.com/pypi-13.1/numpy/numpy-1.26.2-0-cp311-cp311-android_21_arm64_v8a.whl (5.0 MB)
Collecting regex
  Using cached https://chaquo.com/pypi-13.1/regex/regex-2023.10.3-0-cp311-cp311-android_21_arm64_v8a.whl (284 kB)
Collecting sentencepiece
  Using cached sentencepiece-0.2.0.tar.gz (2.6 MB)
Collecting indic-nlp-library
  Using cached indic_nlp_library-0.92-py3-none-any.whl (40 kB)
Processing /Users/gireeshachowdhary/Library/Caches/chaquopy/pip/wheels/34/cc/6e/444b747bf2adc6a0530acd6a7afda9f908b219906cfd4df2e7/mosestokenizer-1.2.1-py3-none-any.whl
Collecting nltk
  Using cached nltk-3.9.1-py3-none-any.whl (1.5 MB)
Collecting sacremoses
  Using cached sacremoses-0.1.1-py3-none-any.whl (897 kB)
Processing /Users/gireeshachowdhary/Desktop/C4GT-Bhashini/NMT/beeware_app/nmtbhashini/src/nmtbhashini/resources/ctranslate2_lib-4.1.0-0-py3-none-android_21_arm64_v8a.whl
Processing /Users/gireeshachowdhary/Desktop/C4GT-Bhashini/NMT/beeware_app/nmtbhashini/src/nmtbhashini/resources/ctranslate2_python-4.1.0-0-py3-none-android_21_arm64_v8a.whl
Collecting toga-android~=0.4.5
  Using cached toga_android-0.4.7-py3-none-any.whl (67 kB)
Collecting chaquopy-libcxx>=11000
  Using cached https://chaquo.com/pypi-13.1/chaquopy-libcxx/chaquopy_libcxx-11000-0-py3-none-android_21_arm64_v8a.whl (308 kB)
Collecting chaquopy-openblas>=0.2.20
  Using cached https://chaquo.com/pypi-13.1/chaquopy-openblas/chaquopy_openblas-0.2.20-5-py3-none-android_21_arm64_v8a.whl (4.3 MB)
Collecting sphinx-argparse
  Using cached sphinx_argparse-0.5.2-py3-none-any.whl (12 kB)
Collecting pandas
  Using cached https://chaquo.com/pypi-13.1/pandas/pandas-1.5.0-1-cp311-cp311-android_21_arm64_v8a.whl (10.1 MB)
Collecting morfessor
  Using cached Morfessor-2.0.6-py3-none-any.whl (35 kB)
Collecting sphinx-rtd-theme
  Downloading sphinx_rtd_theme-3.0.1-py2.py3-none-any.whl (7.7 MB)
Processing /Users/gireeshachowdhary/Library/Caches/chaquopy/pip/wheels/ad/d6/21/763477f07e94a9d2567147f91d34897dfd1eb1062350302969/uctools-1.3.0-py3-none-any.whl
Processing /Users/gireeshachowdhary/Library/Caches/chaquopy/pip/wheels/28/b0/06/bccd361b5b8bbf1ec88b72164fa52daaaa2221865fb980dff1/toolwrapper-2.1.0-py3-none-any.whl
Processing /Users/gireeshachowdhary/Library/Caches/chaquopy/pip/wheels/1a/b0/8c/4b75c4116c31f83c8f9f047231251e13cc74481cca4a78a9ce/docopt-0.6.2-py2.py3-none-any.whl
Collecting openfile
  Using cached openfile-0.0.7-py3-none-any.whl (2.4 kB)
Collecting joblib
  Using cached joblib-1.4.2-py3-none-any.whl (301 kB)
Collecting tqdm
  Using cached tqdm-4.66.5-py3-none-any.whl (78 kB)
Collecting click
  Using cached click-8.1.7-py3-none-any.whl (97 kB)
Collecting toga-core==0.4.7
  Using cached toga_core-0.4.7-py3-none-any.whl (139 kB)
Collecting chaquopy-libgfortran>=4.9
  Using cached https://chaquo.com/pypi-13.1/chaquopy-libgfortran/chaquopy_libgfortran-4.9-0-py3-none-android_21_arm64_v8a.whl (495 kB)
Collecting docutils>=0.19
  Using cached docutils-0.21.2-py3-none-any.whl (587 kB)
Collecting sphinx>=5.1.0
  Downloading sphinx-8.1.0-py3-none-any.whl (3.5 MB)
Collecting pytz>=2020.1
  Using cached pytz-2024.2-py2.py3-none-any.whl (508 kB)
Collecting python-dateutil>=2.8.1
  Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB)
Collecting sphinxcontrib-jquery<5,>=4
  Using cached sphinxcontrib_jquery-4.1-py2.py3-none-any.whl (121 kB)
Collecting travertino>=0.3.0
  Using cached travertino-0.3.0-py3-none-any.whl (16 kB)
Collecting sphinxcontrib-htmlhelp>=2.0.6
  Using cached sphinxcontrib_htmlhelp-2.1.0-py3-none-any.whl (98 kB)
Collecting Jinja2>=3.1
  Using cached jinja2-3.1.4-py3-none-any.whl (133 kB)
Collecting Pygments>=2.17
  Using cached pygments-2.18.0-py3-none-any.whl (1.2 MB)
Collecting imagesize>=1.3
  Using cached imagesize-1.4.1-py2.py3-none-any.whl (8.8 kB)
Collecting sphinxcontrib-jsmath>=1.0.1
  Using cached sphinxcontrib_jsmath-1.0.1-py2.py3-none-any.whl (5.1 kB)
Collecting alabaster>=0.7.14
  Using cached alabaster-1.0.0-py3-none-any.whl (13 kB)
Collecting requests>=2.30.0
  Using cached requests-2.32.3-py3-none-any.whl (64 kB)
Collecting sphinxcontrib-qthelp>=1.0.6
  Using cached sphinxcontrib_qthelp-2.0.0-py3-none-any.whl (88 kB)
Collecting snowballstemmer>=2.2
  Using cached snowballstemmer-2.2.0-py2.py3-none-any.whl (93 kB)
Collecting sphinxcontrib-applehelp>=1.0.7
  Using cached sphinxcontrib_applehelp-2.0.0-py3-none-any.whl (119 kB)
Collecting sphinxcontrib-devhelp>=1.0.6
  Using cached sphinxcontrib_devhelp-2.0.0-py3-none-any.whl (82 kB)
Collecting sphinxcontrib-serializinghtml>=1.1.9
  Using cached sphinxcontrib_serializinghtml-2.0.0-py3-none-any.whl (92 kB)
Collecting babel>=2.13
  Using cached babel-2.16.0-py3-none-any.whl (9.6 MB)
Collecting packaging>=23.0
  Using cached packaging-24.1-py3-none-any.whl (53 kB)
Collecting six>=1.5
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting MarkupSafe>=2.0
  Downloading markupsafe-3.0.1.tar.gz (20 kB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
    Preparing wheel metadata: started
    Preparing wheel metadata: finished with status 'done'
Collecting certifi>=2017.4.17
  Using cached certifi-2024.8.30-py3-none-any.whl (167 kB)
Collecting urllib3<3,>=1.21.1
  Using cached urllib3-2.2.3-py3-none-any.whl (126 kB)
Collecting idna<4,>=2.5
  Using cached idna-3.10-py3-none-any.whl (70 kB)
Collecting charset-normalizer<4,>=2
  Downloading charset_normalizer-3.4.0-py3-none-any.whl (49 kB)
Building wheels for collected packages: sentencepiece, MarkupSafe
  Building wheel for sentencepiece (setup.py): started
  Building wheel for sentencepiece (setup.py): finished with status 'error'
  ERROR: Command errored out with exit status 1:
   command: /Users/gireeshachowdhary/Desktop/C4GT-Bhashini/NMT/beeware_app/nmtbhashini/build/nmtbhashini/android/gradle/app/build/python/env/debug/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/vg/68_c7bhx0c79x9t4k94_6hvm0000gn/T/pip-install-_8vz4ym6/sentencepiece/setup.py'"'"'; __file__='"'"'/private/var/folders/vg/68_c7bhx0c79x9t4k94_6hvm0000gn/T/pip-install-_8vz4ym6/sentencepiece/setup.py'"'"';import chaquopy_monkey; chaquopy_monkey.disable_native();f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/vg/68_c7bhx0c79x9t4k94_6hvm0000gn/T/pip-wheel-3ydfyrpi
       cwd: /private/var/folders/vg/68_c7bhx0c79x9t4k94_6hvm0000gn/T/pip-install-_8vz4ym6/sentencepiece/
  Complete output (85 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.macosx-10.9-universal2-cpython-311
  creating build/lib.macosx-10.9-universal2-cpython-311/sentencepiece
  copying src/sentencepiece/__init__.py -> build/lib.macosx-10.9-universal2-cpython-311/sentencepiece
  copying src/sentencepiece/_version.py -> build/lib.macosx-10.9-universal2-cpython-311/sentencepiece
  copying src/sentencepiece/sentencepiece_model_pb2.py -> build/lib.macosx-10.9-universal2-cpython-311/sentencepiece
  copying src/sentencepiece/sentencepiece_pb2.py -> build/lib.macosx-10.9-universal2-cpython-311/sentencepiece
  running build_ext
  Package sentencepiece was not found in the pkg-config search path.
  Perhaps you should add the directory containing `sentencepiece.pc'
  to the PKG_CONFIG_PATH environment variable
  No package 'sentencepiece' found
  CMake Deprecation Warning at CMakeLists.txt:15 (cmake_minimum_required):
    Compatibility with CMake < 3.5 will be removed from a future version of
    CMake.

    Update the VERSION argument <min> value or use a ...<max> suffix to tell
    CMake that the project does not need compatibility with older versions.

  -- VERSION: 0.2.0
  CMake Error at /opt/homebrew/Cellar/cmake/3.30.4/share/cmake/Modules/CMakeDetermineCCompiler.cmake:49 (message):
    Could not find compiler set in environment variable CC:

    Chaquopy_cannot_compile_native_code.
  Call Stack (most recent call first):
    CMakeLists.txt:23 (project)

  CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage
  CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage
  -- Configuring incomplete, errors occurred!
  ./build_bundled.sh: line 22: nproc: command not found
  make: Makefile: No such file or directory
  make: *** No rule to make target `Makefile'.  Stop.
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/private/var/folders/vg/68_c7bhx0c79x9t4k94_6hvm0000gn/T/pip-install-_8vz4ym6/sentencepiece/setup.py", line 169, in <module>
      setup(
    File "/Users/gireeshachowdhary/Desktop/C4GT-Bhashini/NMT/beeware_app/nmtbhashini/build/nmtbhashini/android/gradle/app/build/python/env/debug/lib/python3.11/site-packages/setuptools/__init__.py", line 103, in setup
      return distutils.core.setup(**attrs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/gireeshachowdhary/Desktop/C4GT-Bhashini/NMT/beeware_app/nmtbhashini/build/nmtbhashini/android/gradle/app/build/python/env/debug/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 185, in setup
      return run_commands(dist)
             ^^^^^^^^^^^^^^^^^^
    File "/Users/gireeshachowdhary/Desktop/C4GT-Bhashini/NMT/beeware_app/nmtbhashini/build/nmtbhashini/android/gradle/app/build/python/env/debug/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
      dist.run_commands()
    File "/Users/gireeshachowdhary/Desktop/C4GT-Bhashini/NMT/beeware_app/nmtbhashini/build/nmtbhashini/android/gradle/app/build/python/env/debug/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
      self.run_command(cmd)
    File "/Users/gireeshachowdhary/Desktop/C4GT-Bhashini/NMT/beeware_app/nmtbhashini/build/nmtbhashini/android/gradle/app/build/python/env/debug/lib/python3.11/site-packages/setuptools/dist.py", line 989, in run_command
      super().run_command(command)
    File "/Users/gireeshachowdhary/Desktop/C4GT-Bhashini/NMT/beeware_app/nmtbhashini/build/nmtbhashini/android/gradle/app/build/python/env/debug/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/Users/gireeshachowdhary/Desktop/C4GT-Bhashini/NMT/beeware_app/nmtbhashini/build/nmtbhashini/android/gradle/app/build/python/env/debug/lib/python3.11/site-packages/wheel/bdist_wheel.py", line 299, in run
      self.run_command('build')
    File "/Users/gireeshachowdhary/Desktop/C4GT-Bhashini/NMT/beeware_app/nmtbhashini/build/nmtbhashini/android/gradle/app/build/python/env/debug/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
      self.distribution.run_command(command)
    File "/Users/gireeshachowdhary/Desktop/C4GT-Bhashini/NMT/beeware_app/nmtbhashini/build/nmtbhashini/android/gradle/app/build/python/env/debug/lib/python3.11/site-packages/setuptools/dist.py", line 989, in run_command
      super().run_command(command)
    File "/Users/gireeshachowdhary/Desktop/C4GT-Bhashini/NMT/beeware_app/nmtbhashini/build/nmtbhashini/android/gradle/app/build/python/env/debug/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/Users/gireeshachowdhary/Desktop/C4GT-Bhashini/NMT/beeware_app/nmtbhashini/build/nmtbhashini/android/gradle/app/build/python/env/debug/lib/python3.11/site-packages/setuptools/_distutils/command/build.py", line 131, in run
      self.run_command(cmd_name)
    File "/Users/gireeshachowdhary/Desktop/C4GT-Bhashini/NMT/beeware_app/nmtbhashini/build/nmtbhashini/android/gradle/app/build/python/env/debug/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
      self.distribution.run_command(command)
    File "/Users/gireeshachowdhary/Desktop/C4GT-Bhashini/NMT/beeware_app/nmtbhashini/build/nmtbhashini/android/gradle/app/build/python/env/debug/lib/python3.11/site-packages/setuptools/dist.py", line 989, in run_command
      super().run_command(command)
    File "/Users/gireeshachowdhary/Desktop/C4GT-Bhashini/NMT/beeware_app/nmtbhashini/build/nmtbhashini/android/gradle/app/build/python/env/debug/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/Users/gireeshachowdhary/Desktop/C4GT-Bhashini/NMT/beeware_app/nmtbhashini/build/nmtbhashini/android/gradle/app/build/python/env/debug/lib/python3.11/site-packages/setuptools/command/build_ext.py", line 88, in run
      _build_ext.run(self)
    File "/Users/gireeshachowdhary/Desktop/C4GT-Bhashini/NMT/beeware_app/nmtbhashini/build/nmtbhashini/android/gradle/app/build/python/env/debug/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
      self.build_extensions()
    File "/Users/gireeshachowdhary/Desktop/C4GT-Bhashini/NMT/beeware_app/nmtbhashini/build/nmtbhashini/android/gradle/app/build/python/env/debug/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 467, in build_extensions
      self._build_extensions_serial()
    File "/Users/gireeshachowdhary/Desktop/C4GT-Bhashini/NMT/beeware_app/nmtbhashini/build/nmtbhashini/android/gradle/app/build/python/env/debug/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 493, in _build_extensions_serial
      self.build_extension(ext)
    File "/private/var/folders/vg/68_c7bhx0c79x9t4k94_6hvm0000gn/T/pip-install-_8vz4ym6/sentencepiece/setup.py", line 87, in build_extension
      subprocess.check_call(['./build_bundled.sh', __version__])
    File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/subprocess.py", line 413, in check_call
      raise CalledProcessError(retcode, cmd)
  subprocess.CalledProcessError: Command '['./build_bundled.sh', '0.2.0']' returned non-zero exit status 2.
  ----------------------------------------
ERROR: Failed to install sentencepiece from https://files.pythonhosted.org/packages/c9/d2/b9c7ca067c26d8ff085d252c89b5f69609ca93fb85a00ede95f4857865d4/sentencepiece-0.2.0.tar.gz#sha256=a52c19171daaf2e697dc6cbe67684e0fa341b1248966f6aebb541de654d15843 (from -r requirements.txt (line 4)).
For assistance, please raise an issue at https://github.com/chaquo/chaquopy/issues.
Chaquopy: Exit status 1

> Task :app:generateDebugPythonRequirements FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:generateDebugPythonRequirements'.
> Process 'command '/Users/gireeshachowdhary/Desktop/C4GT-Bhashini/NMT/beeware_app/nmtbhashini/build/nmtbhashini/android/gradle/app/build/python/env/debug/bin/python'' finished with non-zero exit value 1

  To view full details in Android Studio:
  * Click the 'Build: failed' caption to the left of this message.
  * Then scroll up to see the full output.

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
> Get more help at https://help.gradle.org.

BUILD FAILED in 38s
2 actionable tasks: 1 executed, 1 up-to-date
Building... errored

Error while building project.

Log saved to /Users/gireeshachowdhary/Desktop/C4GT-Bhashini/NMT/beeware_app/nmtbhashini/logs/briefcase.2024_10_11-01_49_40.build.log

I have also provided path to locally built ctranslate2 packages in pyproject.toml file. Also when i was building android package wheel for sentencepiece, it was showing me this error : - 

build-wheel: cd /home/aryan/chaquopy/server/pypi/packages/sentencepiece/build/0.1.95
build-wheel: rm -rf /home/aryan/chaquopy/server/pypi/packages/sentencepiece/build/0.1.95/cp311-cp311-android_21_arm64_v8a
build-wheel: mkdir -p /home/aryan/chaquopy/server/pypi/packages/sentencepiece/build/0.1.95/cp311-cp311-android_21_arm64_v8a
build-wheel: Using cached sdist
build-wheel: tar -C /tmp/build-wheel-3m2mpa7h -xf sentencepiece-0.1.95.tar.gz
build-wheel: mv /tmp/build-wheel-3m2mpa7h/sentencepiece-0.1.95 /home/aryan/chaquopy/server/pypi/packages/sentencepiece/build/0.1.95/cp311-cp311-android_21_arm64_v8a/src
build-wheel: rm -rf /tmp/build-wheel-3m2mpa7h
build-wheel: cd /home/aryan/chaquopy/server/pypi/packages/sentencepiece/build/0.1.95/cp311-cp311-android_21_arm64_v8a/src
build-wheel: patch -p1 -i /home/aryan/chaquopy/server/pypi/packages/sentencepiece/patches/chaquopy.patch
patching file setup.py
patching file build_bundled.sh
build-wheel: mkdir -p /home/aryan/chaquopy/server/pypi/packages/sentencepiece/build/0.1.95/cp311-cp311-android_21_arm64_v8a/requirements
build-wheel: mkdir -p /home/aryan/chaquopy/server/pypi/packages/sentencepiece/build/0.1.95/cp311-cp311-android_21_arm64_v8a/requirements/chaquopy/include
build-wheel: mkdir -p /home/aryan/chaquopy/server/pypi/packages/sentencepiece/build/0.1.95/cp311-cp311-android_21_arm64_v8a/requirements/chaquopy/lib
build-wheel: unzip -q -d /home/aryan/chaquopy/server/pypi/packages/sentencepiece/build/0.1.95/cp311-cp311-android_21_arm64_v8a/requirements/chaquopy /home/aryan/chaquopy/maven/com/chaquo/python/target/3.11.6-0/target-3.11.6-0-arm64-v8a.zip include/* jniLibs/*
build-wheel: mv /home/aryan/chaquopy/server/pypi/packages/sentencepiece/build/0.1.95/cp311-cp311-android_21_arm64_v8a/requirements/chaquopy/jniLibs/arm64-v8a/* /home/aryan/chaquopy/server/pypi/packages/sentencepiece/build/0.1.95/cp311-cp311-android_21_arm64_v8a/requirements/chaquopy/lib
build-wheel: rm -r /home/aryan/chaquopy/server/pypi/packages/sentencepiece/build/0.1.95/cp311-cp311-android_21_arm64_v8a/requirements/chaquopy/jniLibs
build-wheel: ln -s libpython3.11.so /home/aryan/chaquopy/server/pypi/packages/sentencepiece/build/0.1.95/cp311-cp311-android_21_arm64_v8a/requirements/chaquopy/lib/libpython3..so
build-wheel: ln -s libcrypto_chaquopy.so /home/aryan/chaquopy/server/pypi/packages/sentencepiece/build/0.1.95/cp311-cp311-android_21_arm64_v8a/requirements/chaquopy/lib/libcrypto.so
build-wheel: ln -s libssl_chaquopy.so /home/aryan/chaquopy/server/pypi/packages/sentencepiece/build/0.1.95/cp311-cp311-android_21_arm64_v8a/requirements/chaquopy/lib/libssl.so
build-wheel: ln -s libsqlite3_chaquopy.so /home/aryan/chaquopy/server/pypi/packages/sentencepiece/build/0.1.95/cp311-cp311-android_21_arm64_v8a/requirements/chaquopy/lib/libsqlite3.so
build-wheel: /home/aryan/chaquopy/server/pypi/build/_bootstrap/3.11/bin/pip --version
build-wheel: mkdir -p /home/aryan/chaquopy/server/pypi/packages/sentencepiece/build/0.1.95/cp311-cp311-android_21_arm64_v8a/env
build-wheel: python3.11 -m venv --without-pip /home/aryan/chaquopy/server/pypi/packages/sentencepiece/build/0.1.95/cp311-cp311-android_21_arm64_v8a/env
build-wheel: /home/aryan/chaquopy/server/pypi/build/_bootstrap/3.11/bin/pip --python /home/aryan/chaquopy/server/pypi/packages/sentencepiece/build/0.1.95/cp311-cp311-android_21_arm64_v8a/env/bin/python install cmake==3.28.1 'setuptools >= 40.8.0' wheel
Collecting cmake==3.28.1
  Obtaining dependency information for cmake==3.28.1 from https://files.pythonhosted.org/packages/09/30/df85689d18122becb9b6495cf6778f9ef629bdaa3ec86f49809ab5772e35/cmake-3.28.1-py2.py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata
  Using cached cmake-3.28.1-py2.py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (6.3 kB)
Collecting setuptools>=40.8.0
  Obtaining dependency information for setuptools>=40.8.0 from https://files.pythonhosted.org/packages/ff/ae/f19306b5a221f6a436d8f2238d5b80925004093fa3edea59835b514d9057/setuptools-75.1.0-py3-none-any.whl.metadata
  Using cached setuptools-75.1.0-py3-none-any.whl.metadata (6.9 kB)
Collecting wheel
  Obtaining dependency information for wheel from https://files.pythonhosted.org/packages/1b/d1/9babe2ccaecff775992753d8686970b1e2755d21c8a63be73aba7a4e7d77/wheel-0.44.0-py3-none-any.whl.metadata
  Using cached wheel-0.44.0-py3-none-any.whl.metadata (2.3 kB)
Using cached cmake-3.28.1-py2.py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (26.3 MB)
Using cached setuptools-75.1.0-py3-none-any.whl (1.2 MB)
Using cached wheel-0.44.0-py3-none-any.whl (67 kB)
Installing collected packages: cmake, wheel, setuptools
Successfully installed cmake-3.28.1 setuptools-75.1.0 wheel-0.44.0
build-wheel: abi=arm64-v8a; api_level=21; prefix=/home/aryan/chaquopy/server/pypi/packages/sentencepiece/build/0.1.95/cp311-cp311-android_21_arm64_v8a/requirements/chaquopy; . /home/aryan/chaquopy/server/pypi/../../target/build-common.sh; export
build-wheel: mkdir -p /home/aryan/chaquopy/server/pypi/packages/sentencepiece/build/0.1.95/cp311-cp311-android_21_arm64_v8a/wrappers
/home/aryan/chaquopy/server/pypi/packages/sentencepiece/build/0.1.95/cp311-cp311-android_21_arm64_v8a/env/lib/python3.11/site-packages/setuptools/dist.py:452: SetuptoolsDeprecationWarning: Invalid dash-separated options
!!

        ********************************************************************************
        Usage of dash-separated 'description-file' will not be supported in future
        versions. Please use the underscore name 'description_file' instead.

        This deprecation is overdue, please update your project and remove deprecated
        calls to avoid build errors in the future.

        See https://setuptools.pypa.io/en/latest/userguide/declarative_config.html for details.
        ********************************************************************************

!!
  opt = self.warn_dash_deprecation(opt, section)
/home/aryan/chaquopy/server/pypi/packages/sentencepiece/build/0.1.95/cp311-cp311-android_21_arm64_v8a/env/lib/python3.11/site-packages/setuptools/_distutils/dist.py:261: UserWarning: Unknown distribution option: 'test_suite'
  warnings.warn(msg)
running egg_info
writing src/sentencepiece.egg-info/PKG-INFO
writing dependency_links to src/sentencepiece.egg-info/dependency_links.txt
writing top-level names to src/sentencepiece.egg-info/top_level.txt
reading manifest file 'src/sentencepiece.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'src/sentencepiece.egg-info/SOURCES.txt'
build-wheel: abi=arm64-v8a; api_level=21; prefix=/home/aryan/chaquopy/server/pypi/packages/sentencepiece/build/0.1.95/cp311-cp311-android_21_arm64_v8a/requirements/chaquopy; . /home/aryan/chaquopy/server/pypi/../../target/build-common.sh; export
build-wheel: /home/aryan/android-sdk/ndk/22.1.7171670/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-ar rc /home/aryan/chaquopy/server/pypi/packages/sentencepiece/build/0.1.95/cp311-cp311-android_21_arm64_v8a/requirements/chaquopy/lib/libpthread.a
build-wheel: /home/aryan/android-sdk/ndk/22.1.7171670/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-ar rc /home/aryan/chaquopy/server/pypi/packages/sentencepiece/build/0.1.95/cp311-cp311-android_21_arm64_v8a/requirements/chaquopy/lib/librt.a
build-wheel: Building with PEP 517
/home/aryan/chaquopy/server/pypi/packages/sentencepiece/build/0.1.95/cp311-cp311-android_21_arm64_v8a/env/lib/python3.11/site-packages/setuptools/_distutils/dist.py:261: UserWarning: Unknown distribution option: 'test_suite'
  warnings.warn(msg)
/home/aryan/chaquopy/server/pypi/packages/sentencepiece/build/0.1.95/cp311-cp311-android_21_arm64_v8a/env/lib/python3.11/site-packages/setuptools/dist.py:452: SetuptoolsDeprecationWarning: Invalid dash-separated options
!!

        ********************************************************************************
        Usage of dash-separated 'description-file' will not be supported in future
        versions. Please use the underscore name 'description_file' instead.

        This deprecation is overdue, please update your project and remove deprecated
        calls to avoid build errors in the future.

        See https://setuptools.pypa.io/en/latest/userguide/declarative_config.html for details.
        ********************************************************************************

!!
  opt = self.warn_dash_deprecation(opt, section)
running bdist_wheel
running build
running build_py
creating build/lib.linux_aarch64-cpython-311/sentencepiece
copying src/sentencepiece/__init__.py -> build/lib.linux_aarch64-cpython-311/sentencepiece
copying src/sentencepiece/sentencepiece_model_pb2.py -> build/lib.linux_aarch64-cpython-311/sentencepiece
copying src/sentencepiece/sentencepiece_pb2.py -> build/lib.linux_aarch64-cpython-311/sentencepiece
running build_ext
Package sentencepiece was not found in the pkg-config search path.
Perhaps you should add the directory containing `sentencepiece.pc'
to the PKG_CONFIG_PATH environment variable
No package 'sentencepiece' found
Cloning into 'sentencepiece'...
remote: Enumerating objects: 264, done.
remote: Counting objects: 100% (264/264), done.
remote: Compressing objects: 100% (238/238), done.
remote: Total 264 (delta 57), reused 59 (delta 17), pack-reused 0 (from 0)
Receiving objects: 100% (264/264), 5.65 MiB | 6.00 MiB/s, done.
Resolving deltas: 100% (57/57), done.
Note: switching to '0e6dfbf86e2fa6d86a3d9a8a08a628da71c073e0'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

patching file src/CMakeLists.txt
patching file src/util.h
CMake Deprecation Warning at CMakeLists.txt:15 (cmake_minimum_required):
  Compatibility with CMake < 3.5 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.

-- VERSION: 0.1.95
-- The C compiler identification is Clang 11.0.5
-- The CXX compiler identification is Clang 11.0.5
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /home/aryan/android-sdk/ndk/22.1.7171670/toolchains/llvm/prebuilt/linux-x86_64/bin/clang - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /home/aryan/android-sdk/ndk/22.1.7171670/toolchains/llvm/prebuilt/linux-x86_64/bin/clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - not found
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Found Threads: TRUE  
-- Not Found TCMalloc: TCMALLOC_LIB-NOTFOUND
-- Configuring done (0.7s)
-- Generating done (0.0s)
-- Build files have been written to: /home/aryan/chaquopy/server/pypi/packages/sentencepiece/build/0.1.95/cp311-cp311-android_21_arm64_v8a/src/bundled/sentencepiece/build
[  1%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/arenastring.cc.o
[  4%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/bytestream.cc.o
[  4%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/arena.cc.o
[  6%] Building CXX object src/CMakeFiles/sentencepiece_train-static.dir/builder.cc.o
[  7%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/coded_stream.cc.o
[  9%] Building CXX object src/CMakeFiles/sentencepiece_train-static.dir/unicode_script.cc.o
[ 11%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/common.cc.o
/home/aryan/chaquopy/server/pypi/packages/sentencepiece/build/0.1.95/cp311-cp311-android_21_arm64_v8a/src/bundled/sentencepiece/src/builder.cc:47:15: warning: unused variable 'kMaxUnicode' [-Wunused-const-variable]
constexpr int kMaxUnicode = 0x10FFFF;
              ^
/home/aryan/chaquopy/server/pypi/packages/sentencepiece/build/0.1.95/cp311-cp311-android_21_arm64_v8a/src/bundled/sentencepiece/src/builder.cc:49:23: warning: unused variable 'kDefaultNormalizerName' [-Wunused-const-variable]
static constexpr char kDefaultNormalizerName[] = "nfkc";
                      ^
[ 12%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/extension_set.cc.o
[ 14%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/generated_enum_util.cc.o
[ 15%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/generated_message_table_driven_lite.cc.o
2 warnings generated.
[ 17%] Building CXX object src/CMakeFiles/sentencepiece_train-static.dir/trainer_factory.cc.o
[ 19%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/generated_message_util.cc.o
[ 20%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/implicit_weak_message.cc.o
[ 22%] Building CXX object src/CMakeFiles/sentencepiece_train-static.dir/trainer_interface.cc.o
[ 23%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/int128.cc.o
[ 25%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/io_win32.cc.o
[ 26%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/message_lite.cc.o
[ 28%] Building CXX object src/CMakeFiles/sentencepiece_train-static.dir/unigram_model_trainer.cc.o
[ 30%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/parse_context.cc.o
[ 31%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/repeated_field.cc.o
[ 33%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/status.cc.o
[ 34%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/statusor.cc.o
[ 36%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/stringpiece.cc.o
[ 38%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/stringprintf.cc.o
[ 39%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/structurally_valid.cc.o
[ 41%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/strutil.cc.o
[ 42%] Building CXX object src/CMakeFiles/sentencepiece_train-static.dir/word_model_trainer.cc.o
[ 44%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/time.cc.o
[ 46%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/wire_format_lite.cc.o
[ 47%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/zero_copy_stream.cc.o
[ 49%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/zero_copy_stream_impl.cc.o
[ 50%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/zero_copy_stream_impl_lite.cc.o
[ 52%] Building CXX object src/CMakeFiles/sentencepiece_train-static.dir/char_model_trainer.cc.o
[ 53%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/builtin_pb/sentencepiece.pb.cc.o
[ 55%] Building CXX object src/CMakeFiles/sentencepiece_train-static.dir/bpe_model_trainer.cc.o
[ 57%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/builtin_pb/sentencepiece_model.pb.cc.o
[ 58%] Building CXX object src/CMakeFiles/sentencepiece_train-static.dir/sentencepiece_trainer.cc.o
[ 60%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/bpe_model.cc.o
[ 61%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/char_model.cc.o
[ 63%] Building CXX object src/CMakeFiles/sentencepiece_train-static.dir/pretokenizer_for_training.cc.o
[ 65%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/error.cc.o
[ 66%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/filesystem.cc.o
[ 68%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/model_factory.cc.o
[ 69%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/model_interface.cc.o
[ 71%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/normalizer.cc.o
[ 73%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/sentencepiece_processor.cc.o
[ 74%] Linking CXX static library libsentencepiece_train.a
[ 74%] Built target sentencepiece_train-static
[ 76%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/unigram_model.cc.o
[ 77%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/util.cc.o
[ 79%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/word_model.cc.o
[ 80%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/__/third_party/absl/strings/string_view.cc.o
[ 82%] Building CXX object src/CMakeFiles/sentencepiece-static.dir/__/third_party/absl/flags/flag.cc.o
[ 84%] Linking CXX static library libsentencepiece.a
[ 84%] Built target sentencepiece-static
[ 85%] Building CXX object src/CMakeFiles/spm_encode.dir/spm_encode_main.cc.o
[ 87%] Building CXX object src/CMakeFiles/spm_decode.dir/spm_decode_main.cc.o
[ 88%] Building CXX object src/CMakeFiles/spm_train.dir/spm_train_main.cc.o
[ 90%] Building CXX object src/CMakeFiles/spm_normalize.dir/spm_normalize_main.cc.o
[ 92%] Linking CXX executable spm_normalize
[ 93%] Linking CXX executable spm_train
[ 93%] Built target spm_normalize
[ 95%] Building CXX object src/CMakeFiles/spm_export_vocab.dir/spm_export_vocab_main.cc.o
[ 95%] Built target spm_train
[ 96%] Linking CXX executable spm_decode
[ 96%] Built target spm_decode
[ 98%] Linking CXX executable spm_export_vocab
[ 98%] Built target spm_export_vocab
[100%] Linking CXX executable spm_encode
[100%] Built target spm_encode
[ 66%] Built target sentencepiece-static
[ 84%] Built target sentencepiece_train-static
[ 87%] Built target spm_encode
[ 90%] Built target spm_decode
[ 93%] Built target spm_normalize
[ 96%] Built target spm_train
[100%] Built target spm_export_vocab
Install the project...
-- Install configuration: ""
-- Installing: /home/aryan/chaquopy/server/pypi/packages/sentencepiece/build/0.1.95/cp311-cp311-android_21_arm64_v8a/src/bundled/lib/pkgconfig/sentencepiece.pc
-- Installing: /home/aryan/chaquopy/server/pypi/packages/sentencepiece/build/0.1.95/cp311-cp311-android_21_arm64_v8a/src/bundled/lib/libsentencepiece.a
-- Installing: /home/aryan/chaquopy/server/pypi/packages/sentencepiece/build/0.1.95/cp311-cp311-android_21_arm64_v8a/src/bundled/lib/libsentencepiece_train.a
-- Installing: /home/aryan/chaquopy/server/pypi/packages/sentencepiece/build/0.1.95/cp311-cp311-android_21_arm64_v8a/src/bundled/bin/spm_encode
-- Installing: /home/aryan/chaquopy/server/pypi/packages/sentencepiece/build/0.1.95/cp311-cp311-android_21_arm64_v8a/src/bundled/bin/spm_decode
-- Installing: /home/aryan/chaquopy/server/pypi/packages/sentencepiece/build/0.1.95/cp311-cp311-android_21_arm64_v8a/src/bundled/bin/spm_normalize
-- Installing: /home/aryan/chaquopy/server/pypi/packages/sentencepiece/build/0.1.95/cp311-cp311-android_21_arm64_v8a/src/bundled/bin/spm_train
-- Installing: /home/aryan/chaquopy/server/pypi/packages/sentencepiece/build/0.1.95/cp311-cp311-android_21_arm64_v8a/src/bundled/bin/spm_export_vocab
-- Installing: /home/aryan/chaquopy/server/pypi/packages/sentencepiece/build/0.1.95/cp311-cp311-android_21_arm64_v8a/src/bundled/include/sentencepiece_trainer.h
-- Installing: /home/aryan/chaquopy/server/pypi/packages/sentencepiece/build/0.1.95/cp311-cp311-android_21_arm64_v8a/src/bundled/include/sentencepiece_processor.h
## cflags=-std=c++11 -I./bundled/include
## libs=-L./bundled/lib -lsentencepiece -lsentencepiece_train
building 'sentencepiece._sentencepiece' extension
creating build/temp.linux_aarch64-cpython-311/src/sentencepiece
/home/aryan/chaquopy/server/pypi/packages/sentencepiece/build/0.1.95/cp311-cp311-android_21_arm64_v8a/wrappers/aarch64-linux-android21-clang++ -fPIC -I/home/aryan/chaquopy/server/pypi/packages/sentencepiece/build/0.1.95/cp311-cp311-android_21_arm64_v8a/env/include -I/home/aryan/anaconda3/envs/build-wheel/include/python3.11 -c src/sentencepiece/sentencepiece_wrap.cxx -o build/temp.linux_aarch64-cpython-311/src/sentencepiece/sentencepiece_wrap.o -std=c++11 -I./bundled/include
Chaquopy: ignored invalid -I directory: '/home/aryan/anaconda3/envs/build-wheel/include/python3.11'
src/sentencepiece/sentencepiece_wrap.cxx:178:11: fatal error: 'Python.h' file not found
# include <Python.h>
          ^~~~~~~~~~
1 error generated.
error: command '/home/aryan/chaquopy/server/pypi/packages/sentencepiece/build/0.1.95/cp311-cp311-android_21_arm64_v8a/wrappers/aarch64-linux-android21-clang++' failed with exit code 1
build-wheel: Error: Backend subprocess exited when trying to invoke build_wheel

Please can some guide me about how to go around this?

mhsmith commented 1 month ago

We currently only have Android builds of sentencepiece for Python 3.8. To change the Python version of your app:

If anyone else needs this package for newer Python versions, please click the thumbs up button above.