Unable to build on Android Studio due to scikit-learn Error

ys31jp commented 2 years ago

Chaquopy version: 12.0.0

Python version: 3.8.13

Relevant parts of your build.gradle file

scikit-learn-0.24.1

Describe your issue

FileNotFoundError: [WinError 3] The system cannot find the path specified: android\app\build\pip\debug\armeabi-v7a\sklearn\datasets\tests\data\openml\292\api-v1-json-data-list-data_name-australian-limit-2-data_version-1-status-deactivated.json.gz'

left side of the images is folder on the Android studio. Meanwhile, on the right side is anaconda environment build with pip.

> Task :app:generateDebugPythonRequirements
Chaquopy: Installing for arm64-v8a
Looking in indexes: https://pypi.org/simple, https://chaquo.com/pypi-7.0
Collecting pandas
  Using version 1.3.2 (newest version is 1.4.2, but Chaquopy prefers native wheels)
  Using cached https://chaquo.com/pypi-7.0/pandas/pandas-1.3.2-0-cp38-cp38-android_21_arm64_v8a.whl
Collecting scikit-learn
  Using version 0.24.1 (newest version is 1.1.1, but Chaquopy prefers native wheels)
  Using cached https://chaquo.com/pypi-7.0/scikit-learn/scikit_learn-0.24.1-0-cp38-cp38-android_21_arm64_v8a.whl
Collecting numpy>=1.17.3 (from pandas)
  Using version 1.19.5 (newest version is 1.22.4, but Chaquopy prefers native wheels)
  Using cached https://chaquo.com/pypi-7.0/numpy/numpy-1.19.5-0-cp38-cp38-android_21_arm64_v8a.whl
Collecting chaquopy-libcxx>=10000 (from pandas)
  Using cached https://chaquo.com/pypi-7.0/chaquopy-libcxx/chaquopy_libcxx-10000-0-py3-none-android_21_arm64_v8a.whl
Collecting python-dateutil>=2.7.3 (from pandas)
  Using cached https://files.pythonhosted.org/packages/36/7a/87837f39d0296e723bb9b62bbb257d0355c7f6128853c78955f57342a56d/python_dateutil-2.8.2-py2.py3-none-any.whl
Collecting pytz>=2017.3 (from pandas)
  Using cached https://files.pythonhosted.org/packages/60/2e/dec1cc18c51b8df33c7c4d0a321b084cf38e1733b98f9d15018880fb4970/pytz-2022.1-py2.py3-none-any.whl
Collecting joblib>=0.11 (from scikit-learn)
  Using cached https://files.pythonhosted.org/packages/3e/d5/0163eb0cfa0b673aa4fe1cd3ea9d8a81ea0f32e50807b0c295871e4aab2e/joblib-1.1.0-py2.py3-none-any.whl
Collecting scipy>=0.19.1 (from scikit-learn)
  Using version 1.4.1 (newest version is 1.8.1, but Chaquopy prefers native wheels)
  Using cached https://chaquo.com/pypi-7.0/scipy/scipy-1.4.1-0-cp38-cp38-android_21_arm64_v8a.whl
Collecting chaquopy-openblas>=0.2.20 (from scikit-learn)
  Using cached https://chaquo.com/pypi-7.0/chaquopy-openblas/chaquopy_openblas-0.2.20-5-py3-none-android_21_arm64_v8a.whl
Collecting threadpoolctl>=2.0.0 (from scikit-learn)
  Using cached https://files.pythonhosted.org/packages/61/cf/6e354304bcb9c6413c4e02a747b600061c21d38ba51e7e544ac7bc66aecc/threadpoolctl-3.1.0-py3-none-any.whl
Collecting chaquopy-libgfortran>=4.9 (from numpy>=1.17.3->pandas)
  Using cached https://chaquo.com/pypi-7.0/chaquopy-libgfortran/chaquopy_libgfortran-4.9-0-py3-none-android_21_arm64_v8a.whl
Collecting six>=1.5 (from python-dateutil>=2.7.3->pandas)
  Using cached https://files.pythonhosted.org/packages/d9/5a/e7c31adbe875f2abbb91bd84cf2dc52d792b5a01506781dbcf25c91daf11/six-1.16.0-py2.py3-none-any.whl
Installing collected packages: chaquopy-libgfortran, chaquopy-openblas, numpy, chaquopy-libcxx, six, python-dateutil, pytz, pandas, joblib, scipy, threadpoolctl, scikit-learn
Successfully installed chaquopy-libcxx-10000 chaquopy-libgfortran-4.9 chaquopy-openblas-0.2.20 joblib-1.1.0 numpy-1.19.5 pandas-1.3.2 python-dateutil-2.8.2 pytz-2022.1 scikit-learn-0.24.1 scipy-1.4.1 six-1.16.0 threadpoolctl-3.1.0
Chaquopy: Installing for armeabi-v7a
Looking in indexes: https://pypi.org/simple, https://chaquo.com/pypi-7.0
Collecting chaquopy-libcxx==10000
  Using cached https://chaquo.com/pypi-7.0/chaquopy-libcxx/chaquopy_libcxx-10000-0-py3-none-android_16_armeabi_v7a.whl
Collecting chaquopy-libgfortran==4.9
  Using cached https://chaquo.com/pypi-7.0/chaquopy-libgfortran/chaquopy_libgfortran-4.9-0-py3-none-android_16_armeabi_v7a.whl
Collecting chaquopy-openblas==0.2.20
  Using cached https://chaquo.com/pypi-7.0/chaquopy-openblas/chaquopy_openblas-0.2.20-5-py3-none-android_16_armeabi_v7a.whl
Collecting numpy==1.19.5
  Using cached https://chaquo.com/pypi-7.0/numpy/numpy-1.19.5-0-cp38-cp38-android_16_armeabi_v7a.whl
Collecting pandas==1.3.2
  Using cached https://chaquo.com/pypi-7.0/pandas/pandas-1.3.2-0-cp38-cp38-android_16_armeabi_v7a.whl
Collecting scikit-learn==0.24.1
  Using cached https://chaquo.com/pypi-7.0/scikit-learn/scikit_learn-0.24.1-0-cp38-cp38-android_16_armeabi_v7a.whl
Collecting scipy==1.4.1
  Using cached https://chaquo.com/pypi-7.0/scipy/scipy-1.4.1-0-cp38-cp38-android_16_armeabi_v7a.whl
Installing collected packages: chaquopy-libcxx, chaquopy-libgfortran, chaquopy-openblas, numpy, pandas, scikit-learn, scipy
Successfully installed chaquopy-libcxx-10000 chaquopy-libgfortran-4.9 chaquopy-openblas-0.2.20 numpy-1.19.5 pandas-1.3.2 scikit-learn-0.24.1 scipy-1.4.1
Chaquopy: Installing for x86
Looking in indexes: https://pypi.org/simple, https://chaquo.com/pypi-7.0
Collecting chaquopy-libcxx==10000
  Using cached https://chaquo.com/pypi-7.0/chaquopy-libcxx/chaquopy_libcxx-10000-0-py3-none-android_16_x86.whl
Collecting chaquopy-libgfortran==4.9
  Using cached https://chaquo.com/pypi-7.0/chaquopy-libgfortran/chaquopy_libgfortran-4.9-0-py3-none-android_16_x86.whl
Collecting chaquopy-openblas==0.2.20
  Using cached https://chaquo.com/pypi-7.0/chaquopy-openblas/chaquopy_openblas-0.2.20-5-py3-none-android_16_x86.whl
Collecting numpy==1.19.5
  Using cached https://chaquo.com/pypi-7.0/numpy/numpy-1.19.5-0-cp38-cp38-android_16_x86.whl
Collecting pandas==1.3.2
  Using cached https://chaquo.com/pypi-7.0/pandas/pandas-1.3.2-0-cp38-cp38-android_16_x86.whl
Collecting scikit-learn==0.24.1
  Using cached https://chaquo.com/pypi-7.0/scikit-learn/scikit_learn-0.24.1-0-cp38-cp38-android_16_x86.whl
Collecting scipy==1.4.1
  Using cached https://chaquo.com/pypi-7.0/scipy/scipy-1.4.1-0-cp38-cp38-android_16_x86.whl
Installing collected packages: chaquopy-libcxx, chaquopy-libgfortran, chaquopy-openblas, numpy, pandas, scikit-learn, scipy
Successfully installed chaquopy-libcxx-10000 chaquopy-libgfortran-4.9 chaquopy-openblas-0.2.20 numpy-1.19.5 pandas-1.3.2 scikit-learn-0.24.1 scipy-1.4.1
Chaquopy: Installing for x86_64
Looking in indexes: https://pypi.org/simple, https://chaquo.com/pypi-7.0
Collecting chaquopy-libcxx==10000
  Using cached https://chaquo.com/pypi-7.0/chaquopy-libcxx/chaquopy_libcxx-10000-0-py3-none-android_21_x86_64.whl
Collecting chaquopy-libgfortran==4.9
  Using cached https://chaquo.com/pypi-7.0/chaquopy-libgfortran/chaquopy_libgfortran-4.9-0-py3-none-android_21_x86_64.whl
Collecting chaquopy-openblas==0.2.20
  Using cached https://chaquo.com/pypi-7.0/chaquopy-openblas/chaquopy_openblas-0.2.20-5-py3-none-android_21_x86_64.whl
Collecting numpy==1.19.5
  Using cached https://chaquo.com/pypi-7.0/numpy/numpy-1.19.5-0-cp38-cp38-android_21_x86_64.whl
Collecting pandas==1.3.2
  Using cached https://chaquo.com/pypi-7.0/pandas/pandas-1.3.2-0-cp38-cp38-android_21_x86_64.whl
Collecting scikit-learn==0.24.1
  Using cached https://chaquo.com/pypi-7.0/scikit-learn/scikit_learn-0.24.1-0-cp38-cp38-android_21_x86_64.whl
Collecting scipy==1.4.1
  Using cached https://chaquo.com/pypi-7.0/scipy/scipy-1.4.1-0-cp38-cp38-android_21_x86_64.whl
Installing collected packages: chaquopy-libcxx, chaquopy-libgfortran, chaquopy-openblas, numpy, pandas, scikit-learn, scipy
Successfully installed chaquopy-libcxx-10000 chaquopy-libgfortran-4.9 chaquopy-openblas-0.2.20 numpy-1.19.5 pandas-1.3.2 scikit-learn-0.24.1 scipy-1.4.1
Traceback (most recent call last):
  File "C:\Users\user\anaconda3\envs\sklearn\lib\shutil.py", line 616, in _rmtree_unsafe
    os.unlink(fullname)
FileNotFoundError: [WinError 3] The system cannot find the path specified: 'C:\\Users\\user\\Documents\\test\\android\\app\\build\\pip\\debug\\armeabi-v7a\\sklearn\\datasets\\tests\\data\\openml\\292\\api-v1-json-data-list-data_name-australian-limit-2-data_version-1-status-deactivated.json.gz'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\user\anaconda3\envs\sklearn\lib\runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\user\anaconda3\envs\sklearn\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\user\Documents\test\android\app\build\generated\python\bp.zip\chaquopy\pip_install.py", line 373, in <module>
  File "C:\Users\user\Documents\test\android\app\build\generated\python\bp.zip\chaquopy\pip_install.py", line 90, in main
  File "C:\Users\user\Documents\test\android\app\build\generated\python\bp.zip\chaquopy\pip_install.py", line 196, in merge_common
  File "C:\Users\user\Documents\test\android\app\build\generated\python\bp.zip\chaquopy\pip_install.py", line 298, in remove
  File "C:\Users\user\Documents\test\android\app\build\generated\python\bp.zip\pip\_internal\utils\misc.py", line 163, in rmtree
  File "C:\Users\user\anaconda3\envs\sklearn\lib\shutil.py", line 740, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "C:\Users\user\anaconda3\envs\sklearn\lib\shutil.py", line 613, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "C:\Users\user\anaconda3\envs\sklearn\lib\shutil.py", line 613, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "C:\Users\user\anaconda3\envs\sklearn\lib\shutil.py", line 613, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "C:\Users\user\anaconda3\envs\sklearn\lib\shutil.py", line 618, in _rmtree_unsafe
    onerror(os.unlink, fullname, sys.exc_info())
  File "C:\Users\user\Documents\test\android\app\build\generated\python\bp.zip\pip\_vendor\retrying.py", line 53, in wrapped_f
  File "C:\Users\user\Documents\test\android\app\build\generated\python\bp.zip\pip\_vendor\retrying.py", line 217, in call
  File "C:\Users\user\Documents\test\android\app\build\generated\python\bp.zip\pip\_vendor\retrying.py", line 252, in get
  File "C:\Users\user\Documents\test\android\app\build\generated\python\bp.zip\pip\_vendor\six.py", line 693, in reraise
  File "C:\Users\user\Documents\test\android\app\build\generated\python\bp.zip\pip\_vendor\retrying.py", line 204, in call
  File "C:\Users\user\Documents\test\android\app\build\generated\python\bp.zip\pip\_internal\utils\misc.py", line 176, in rmtree_errorhandler
FileNotFoundError: [WinError 3] The system cannot find the path specified: 'C:\\Users\\user\\Documents\\test\\android\\app\\build\\pip\\debug\\armeabi-v7a\\sklearn\\datasets\\tests\\data\\openml\\292\\api-v1-json-data-list-data_name-australian-limit-2-data_version-1-status-deactivated.json.gz'

> Task :app:generateDebugPythonRequirements FAILED
Execution failed for task ':app:generateDebugPythonRequirements'.
> Process 'command 'C:/Users/user/anaconda3/envs/sklearn/python.exe'' finished with non-zero exit value 1

ys31jp commented 2 years ago

it happened if the ndk include "x86_64" and 'armeabi-v7a'

ndk { abiFilters 'arm64-v8a', 'x86' //, "x86_64"'armeabi-v7a', }

if commented on the 2, it builds successfully.

mhsmith commented 2 years ago

Thanks for the report. I haven't seen this before, and I can't reproduce it myself.

Can you post the Python-related sections of your build.gradle file?

This is the longest filename in the scikit-learn package, so my first thought was that this might be the Windows 260-character filename limit. But the filename in the error message has a total length of only about 200 characters. Is the path to your app really C:\Users\user\Documents\test\android, or did you replace it with something else for privacy? Either way, it's worth trying to move your project to a location with a shorter path, like directly under your user directory.

If that doesn't help, check whether there's any sign that your anti-virus software has been triggered.

ys31jp commented 2 years ago

Thanks for the report. I haven't seen this before, and I can't reproduce it myself.

Can you post the Python-related sections of your build. Gradle file?

This is the longest filename in the scikit-learn package, so my first thought was that this might be the Windows 260-character filename limit. But the filename in the error message has a total length of only about 200 characters. Is the path to your app really C:\Users\user\Documents\test\android, or did you replace it with something else for privacy? Either way, it's worth trying to move your project to a location with a shorter path, like directly under your user directory.

If that doesn't help, check whether there's any sign that your anti-virus software has been triggered.

Hi, I have checked it and i was caused by the length of the directories and filename. It builed successfully after I change the directories name to shortest.

Thanks a lot for your help.

chaquo / chaquopy