LGTM. Just to clarify for myself, what is rust used for here?
Rust is needed for installation of the listed version of the transformers package. Without, I received error "ERROR: Could not build wheels for tokenizers, which is required to install pyproject.toml-based projects" when attempting install of tokenizers package which transformers depends on.
Rust is needed for installation of the listed version of the transformers package. Without, I received error "ERROR: Could not build wheels for tokenizers, which is required to install pyproject.toml-based projects" when attempting install of tokenizers package which transformers depends on.