Open hasindu2008 opened 8 months ago
Hi @hasindu2008, Have you tried following the instructions in the error message?
Cargo, the Rust package manager, is not installed or is not on PATH.
This package requires Rust and Cargo to compile extensions. Install it through
the system's package manager or via https://rustup.rs/
I am not keen to install another package manager, because I belive an embedded system with a lightweight ARM processor should not be bloated with many dependencies. But I just tried now, yet to end up in another error:
Python 3.8.10
Cloning into 'slow5lib'...
WARNING: The directory '/github/home/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Processing /github/workspace/slow5lib
Requirement already satisfied: numpy in /usr/local/lib/python3.8/dist-packages (from pyslow5==1.2.0b0) (1.24.4)
Building wheels for collected packages: pyslow5
Building wheel for pyslow5 (setup.py): started
Building wheel for pyslow5 (setup.py): still running...
Building wheel for pyslow5 (setup.py): finished with status 'done'
Created wheel for pyslow5: filename=pyslow5-1.2.0b0-cp38-cp38-linux_aarch64.whl size=2364745 sha256=54c1a68dc9a182125ba8f3ce66ce27e72363da75c34fb5390ffe092c18ac7518
Stored in directory: /tmp/pip-ephem-wheel-cache-i5atf7b0/wheels/a8/65/f3/9a470dd09cd16d981c947b80a4b07ba270193ac95329813e57
Successfully built pyslow5
Installing collected packages: pyslow5
Successfully installed pyslow5-1.2.0b0
WARNING: The directory '/github/home/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Processing /github/workspace
Requirement already satisfied: numpy in /usr/local/lib/python3.8/dist-packages (from blue-crab==0.1.0) (1.24.4)
Collecting pod5
Downloading pod5-0.3.2-py3-none-any.whl (69 kB)
Requirement already satisfied: pyslow5>=1.0.1b0 in /usr/local/lib/python3.8/dist-packages (from blue-crab==0.1.0) (1.2.0b0)
Collecting vbz-h5py-plugin
Downloading vbz_h5py_plugin-1.0.1-py3-none-any.whl (1.4 MB)
Collecting tqdm
Downloading tqdm-4.66.1-py3-none-any.whl (78 kB)
Collecting packaging
Downloading packaging-23.2-py3-none-any.whl (53 kB)
Collecting lib-pod5==0.3.2
Downloading lib_pod5-0.3.2-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (2.5 MB)
Collecting polars~=0.19
Downloading polars-0.20.2.tar.gz (3.0 MB)
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Preparing wheel metadata: started
Preparing wheel metadata: finished with status 'done'
Collecting iso8601
Downloading iso8601-2.1.0-py3-none-any.whl (7.5 kB)
Collecting more-itertools
Downloading more_itertools-10.1.0-py3-none-any.whl (55 kB)
Collecting pytz
Downloading pytz-2023.3.post1-py2.py3-none-any.whl (502 kB)
Collecting pyarrow~=14.0.0
Downloading pyarrow-14.0.2-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (36.0 MB)
Collecting h5py~=3.10.0
Downloading h5py-3.10.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (9.0 MB)
Building wheels for collected packages: blue-crab, polars
Building wheel for blue-crab (setup.py): started
Building wheel for blue-crab (setup.py): finished with status 'done'
Created wheel for blue-crab: filename=blue_crab-0.1.0-py3-none-any.whl size=18839 sha256=3f215af733bb2c77e8194da6abb45[1218](https://github.com/Psy-Fer/blue-crab/actions/runs/7286874303/job/19856422254#step:6:1219)870b1a15638378b5f3c7827b7ccbcf4
Stored in directory: /tmp/pip-ephem-wheel-cache-kazqni0j/wheels/a0/95/bc/28c8fad24675c98206b85c82c40eae6717d4e0f16cb0a0e0bc
Building wheel for polars (PEP 517): started
Building wheel for polars (PEP 517): still running...
Building wheel for polars (PEP 517): still running...
Building wheel for polars (PEP 517): still running...
Building wheel for polars (PEP 517): finished with status 'error'
ERROR: Command errored out with exit status 1:
command: /usr/bin/python3 /tmp/tmp_z27vahw build_wheel /tmp/tmpwmlblph8
cwd: /tmp/pip-install-0zjy83x6/polars
Complete output (110 lines):
Running `maturin pep517 build-wheel -i /usr/bin/python3 --compatibility off`
🍹 Building a mixed python/rust project
🔗 Found pyo3 bindings with abi3 support for Python ≥ 3.8
🐍 Not using a specific python interpreter
Compiling libc v0.2.151
Compiling proc-macro2 v1.0.70
Compiling unicode-ident v1.0.12
Compiling autocfg v1.1.0
Compiling cfg-if v1.0.0
Compiling version_check v0.9.4
Compiling serde v1.0.193
Compiling memoffset v0.9.0
Compiling quote v1.0.33
Compiling getrandom v0.2.11
Compiling once_cell v1.19.0
Compiling syn v2.0.41
Compiling jobserver v0.1.27
Compiling crossbeam-utils v0.8.17
Compiling cc v1.0.83
Compiling libm v0.2.8
Compiling ahash v0.8.6
Compiling num-traits v0.2.17
Compiling zerocopy v0.7.31
Compiling pin-project-lite v0.2.13
Compiling crossbeam-epoch v0.9.16
Compiling futures-core v0.3.29
Compiling rayon-core v1.12.0
Compiling pkg-config v0.3.27
Compiling crossbeam-deque v0.8.4
Compiling memchr v2.6.4
Compiling itoa v1.0.10
Compiling either v1.9.0
Compiling slab v0.4.9
Compiling futures-channel v0.3.29
Compiling rayon v1.8.0
Compiling futures-task v0.3.29
Compiling allocator-api2 v0.2.16
Compiling serde_derive v1.0.193
Compiling futures-sink v0.3.29
Compiling ring v0.17.7
Compiling equivalent v1.0.1
Compiling futures-util v0.3.29
Compiling bytes v1.5.0
Compiling hashbrown v0.14.3
error[E0554]: `#![feature]` may not be used on the stable release channel
--> /github/home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/hashbrown-0.14.3/src/lib.rs:15:5
|
15 | / feature(
16 | | test,
17 | | core_intrinsics,
18 | | dropck_eyepatch,
... |
24 | | strict_provenance
25 | | )
| |_____^
error[E0554]: `#![feature]` may not be used on the stable release channel
--> /github/home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/hashbrown-0.14.3/src/lib.rs:16:9
|
16 | test,
| ^^^^
error[E0554]: `#![feature]` may not be used on the stable release channel
--> /github/home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/hashbrown-0.14.3/src/lib.rs:17:9
|
17 | core_intrinsics,
| ^^^^^^^^^^^^^^^
error[E0554]: `#![feature]` may not be used on the stable release channel
--> /github/home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/hashbrown-0.14.3/src/lib.rs:20:9
|
20 | extend_one,
| ^^^^^^^^^^
error[E0554]: `#![feature]` may not be used on the stable release channel
--> /github/home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/hashbrown-0.14.3/src/lib.rs:21:9
|
21 | allocator_api,
| ^^^^^^^^^^^^^
error[E0554]: `#![feature]` may not be used on the stable release channel
--> /github/home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/hashbrown-0.14.3/src/lib.rs:22:9
|
22 | slice_ptr_get,
| ^^^^^^^^^^^^^
error[E0554]: `#![feature]` may not be used on the stable release channel
--> /github/home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/hashbrown-0.14.3/src/lib.rs:23:9
|
23 | maybe_uninit_array_assume_init,
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
error[E0658]: use of unstable library feature 'build_hasher_simple_hash_one'
--> /github/home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/hashbrown-0.14.3/src/map.rs:262:18
|
262 | hash_builder.hash_one(val)
| ^^^^^^^^
|
= note: see issue #86161 <https://github.com/rust-lang/rust/issues/86161> for more information
Compiling futures-macro v0.3.29
Compiling heck v0.4.1
Some errors have detailed explanations: E0554, E0658.
For more information about an error, try `rustc --explain E0554`.
error: could not compile `hashbrown` (lib) due to 8 previous errors
warning: build failed, waiting for other jobs to finish...
💥 maturin failed
Caused by: Failed to build a native library through cargo
Caused by: Cargo build finished with "exit status: 101": `PYO3_ENVIRONMENT_SIGNATURE="cpython-3.8-64bit" PYO3_PYTHON="/usr/bin/python3" PYTHON_SYS_EXECUTABLE="/usr/bin/python3" "cargo" "rustc" "--message-format" "json-render-diagnostics" "--manifest-path" "/tmp/pip-install-0zjy83x6/polars/py-polars/Cargo.toml" "--release" "--lib"`
Error: command ['maturin', 'pep517', 'build-wheel', '-i', '/usr/bin/python3', '--compatibility', 'off'] returned non-zero exit status 1
----------------------------------------
ERROR: Failed building wheel for polars
Successfully built blue-crab
Failed to build polars
ERROR: Could not build wheels for polars which use PEP 517 and cannot be installed directly
Might have something to do with this
https://github.com/nanoporetech/pod5-file-format/issues/94
I hadn't gotten around to fixing it yet.
@Psy-Fer I already fixed that setup.py install to be pip install . on the 21st of December and sent a PR here https://github.com/Psy-Fer/blue-crab/pull/10. So that is not the issue.
polars provides polars-0.19.19-cp38-abi3-manylinux_2_24_aarch64.whl
so you shouldn't need a toolchain for install - see https://pypi.org/project/polars/0.19.19/#files
What OS + libc version is the target machine?
@Psy-Fer I already fixed that setup.py install to be pip install . on the 21st of December and sent a PR here Psy-Fer/blue-crab#10. So that is not the issue.
@hasindu2008, has this been merged in? The PR shows as open and master doesn't have the changes.
Okay I see the failing jobs (OS / libc should be fine) - can you make sure you upgrade pip
as @HalfPhoton suggests. This is often the issue with Python package issues.
oooooooooooooh. The pip install --upgrade pip
?
Let me try 🎉
I might not be doing it right, but seems to still giving a cargo
error:
Python 3.8.10
Cloning into 'slow5lib'...
WARNING: The directory '/github/home/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Processing /github/workspace/slow5lib
Requirement already satisfied: numpy in /usr/local/lib/python3.8/dist-packages (from pyslow5==1.2.0b0) (1.24.4)
Building wheels for collected packages: pyslow5
Building wheel for pyslow5 (setup.py): started
Building wheel for pyslow5 (setup.py): still running...
Building wheel for pyslow5 (setup.py): finished with status 'done'
Created wheel for pyslow5: filename=pyslow5-1.2.0b0-cp38-cp38-linux_aarch64.whl size=2364724 sha256=22e124ac3828ebc8555c6f7b70c4ef05669978eb7437a31bbca1e62c8f59b863
Stored in directory: /tmp/pip-ephem-wheel-cache-uipxecl2/wheels/a8/65/f3/9a470dd09cd16d981c947b80a4b07ba270193ac95329813e57
Successfully built pyslow5
Installing collected packages: pyslow5
Successfully installed pyslow5-1.2.0b0
WARNING: The directory '/github/home/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting pip
Downloading pip-24.0-py3-none-any.whl (2.1 MB)
Installing collected packages: pip
Attempting uninstall: pip
Found existing installation: pip 20.0.2
Not uninstalling pip at /usr/lib/python3/dist-packages, outside environment /usr
Can't uninstall 'pip'. No files were found to uninstall.
Successfully installed pip-24.0
WARNING: The directory '/github/home/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Processing /github/workspace
Requirement already satisfied: numpy in /usr/local/lib/python3.8/dist-packages (from blue-crab==0.1.0) (1.24.4)
Collecting pod5
Downloading pod5-0.3.6-py3-none-any.whl (69 kB)
Requirement already satisfied: pyslow5>=1.0.1b0 in /usr/local/lib/python3.8/dist-packages (from blue-crab==0.1.0) (1.2.0b0)
Collecting polars~=0.19
Downloading polars-0.20.9.tar.gz (3.3 MB)
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Preparing wheel metadata: started
Preparing wheel metadata: finished with status 'error'
ERROR: Command errored out with exit status 1:
command: /usr/bin/python3 /tmp/tmpjw1nfay2 prepare_metadata_for_build_wheel /tmp/tmprnzjb7h8
cwd: /tmp/pip-install-j8uislqq/polars
Complete output (6 lines):
Cargo, the Rust package manager, is not installed or is not on PATH.
This package requires Rust and Cargo to compile extensions. Install it through
the system's package manager or via https://rustup.rs/
Checking for Rust toolchain....
----------------------------------------
ERROR: Command errored out with exit status 1: /usr/bin/python3 /tmp/tmpjw1nfay2 prepare_metadata_for_build_wheel /tmp/tmprnzjb7h8 Check the logs for full command output.
It says Successfully installed pip-24.0
,
Workflow file says:
arm64:
name: pyslow5 ubuntu arm
runs-on: ubuntu-latest
steps:
- run: docker run --rm --privileged multiarch/qemu-user-static:register --reset
- uses: docker://multiarch/ubuntu-core:arm64-focal
with:
args: 'uname -a'
- uses: actions/checkout@v1
- uses: docker://multiarch/ubuntu-core:arm64-focal
with:
args: >
bash -c "apt-get update &&
apt-get install -y zlib1g-dev gcc make python3 python3-pip git wget &&
pip3 install setuptools cython numpy &&
ldd --version && gcc --version && python3 --version &&
git clone -b dev https://github.com/hasindu2008/slow5lib && cd slow5lib && pip3 install . && cd .. &&
pip3 install --upgrade pip && pip3 install . && blue-crab --help && test/test.sh"
@Psy-Fer I already fixed that setup.py install to be pip install . on the 21st of December and sent a PR here Psy-Fer/blue-crab#10. So that is not the issue.
@hasindu2008, has this been merged in? The PR shows as open and master doesn't have the changes.
It is not merged, but the updated workflows are running from this branch: https://github.com/Psy-Fer/blue-crab/tree/linkupdate
Can you try again and add pip3 --version
after the upgrade?
Even though it says Successfully installed pip-24.0
- Not uninstalling pip at /usr/lib/python3/dist-packages, outside environment /usr
is a bit suspicious. 🤞
We are using pod5 pip package in blue-crab to enable BLOW5<->POD5 conversion, however, pod5 installation on ARM64 is failing due to a dependency called polars failing. @Psy-Fer may add more info if required, as I am not too familiar with python package managers.