[1] the official example scripts: (give details below)
[ ] my own modified scripts: (give details below)
A clear and concise description of what the bug is.
When installing the dependencies for layoutlmv3, the tokenizers library throws an error. I found some information suggesting that the issue is related to an improperly configured Rust compilation environment, but I already have Rust version rustc 1.82.0 installed on my machine. The error is as follows:
error: cargo rustc --lib --message-format=json-render-diagnostics --manifest-path Cargo.toml --release -v --features pyo3/extension-module --crate-type cdylib -- failed with code 101
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for tokenizers
Successfully built seqeval deepspeed
Failed to build tokenizers
ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (tokenizers)
To Reproduce
Steps to reproduce the behavior:
Clone the layoutlmv3 repository
Run pip install -r requirements.txt
Encounter a Rust compilation error, preventing the installation of the tokenizers library
Expected behavior
A clear and concise description of what you expected to happen.
Platform:Ubuntu 22.04
Python version: 3.12, 3.11, 3.10 (all versions have issues)
Describe the bug Model I am using ( LayoutLMv3):
The problem arises when using:
A clear and concise description of what the bug is.
When installing the dependencies for layoutlmv3, the tokenizers library throws an error. I found some information suggesting that the issue is related to an improperly configured Rust compilation environment, but I already have Rust version rustc 1.82.0 installed on my machine. The error is as follows:
error: cargo rustc --lib --message-format=json-render-diagnostics --manifest-path Cargo.toml --release -v --features pyo3/extension-module --crate-type cdylib -- failed with code 101 [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for tokenizers Successfully built seqeval deepspeed Failed to build tokenizers ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (tokenizers)
To Reproduce Steps to reproduce the behavior:
Expected behavior A clear and concise description of what you expected to happen.