search

mlverse / tok

Tokenizers from HuggingFace
Other
41 stars 3 forks source link

Cran issues #4

Closed dfalbel closed 1 year ago

dfalbel commented 1 year ago

https://www.stats.ox.ac.uk/pub/bdr/Rblas/MKL/tok.log

* installing *source* package ‘tok’ ...
** package ‘tok’ successfully unpacked and MD5 sums checked
** using staged installation
** libs
using C compiler: ‘gcc-13 (GCC) 13.1.0’
make[2]: Entering directory '/data/gannet/ripley/R/packages/tests-MKL/tok/src'
rm -Rf tok.so ./rust/target/release/libtok.a entrypoint.o
# In some environments, ~/.cargo/bin might not be included in PATH, so we need
# to set it here to ensure cargo can be invoked. It is appended to PATH and
# therefore is only used if cargo is absent from the user's PATH.
export PATH="/data/gannet/ripley/bin:/usr/local/texbin:/usr/local/bin:/data/gannet/ripley/bin:/usr/sbin:/data/gannet/ripley/bin:/usr/local/bin:/usr/local/texbin:/data/gannet/ripley/bin:/usr/sbin:.:/data/gannet/ripley/bin:/usr/local/texbin:/usr/local/bin:/usr/sbin:/usr/lib64/qt-3.3/bin:/usr/share/Modules/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/lib64/openmpi/bin:/usr/lib64/openmpi/bin:/data/gannet/ripley/.cargo/bin" && \
    if [ "" != "true" ]; then \
            export CARGO_HOME=/data/gannet/ripley/R/packages/tests-MKL/tok/src/.cargo; \
    fi && \
    cargo build --lib --release --manifest-path=./rust/Cargo.toml --target-dir ./rust/target
    Updating git repository `https://github.com/extendr/extendr.git`
    Updating crates.io index
gcc-13 -I"/data/gannet/ripley/R/MKL/include" -DNDEBUG   -I/usr/local/include    -fpic  -O2 -Wall -pedantic -mtune=native -Werror=format-security -Wp,-D_FORTIFY_SOURCE=3 -fexceptions -fstack-protector-strong -fstack-clash-protection -fcf-protection -Werror=implicit-function-declaration -Wstrict-prototypes  -c entrypoint.c -o entrypoint.o
 Downloading crates ...
  Downloaded cfg-if v1.0.0
  Downloaded autocfg v1.1.0
  Downloaded bitflags v1.3.2
  Downloaded memoffset v0.9.0
  Downloaded macro_rules_attribute-proc_macro v0.1.3
  Downloaded monostate-impl v0.1.8
  Downloaded darling_macro v0.14.4
  Downloaded fnv v1.0.7
  Downloaded macro_rules_attribute v0.1.3
  Downloaded thiserror-impl v1.0.41
  Downloaded rand_chacha v0.3.1
  Downloaded scopeguard v1.1.0
  Downloaded quote v1.0.29
  Downloaded pkg-config v0.3.27
  Downloaded either v1.8.1
  Downloaded ppv-lite86 v0.2.17
  Downloaded lazy_static v1.4.0
  Downloaded itoa v1.0.8
  Downloaded strsim v0.10.0
  Downloaded rayon-cond v0.1.0
  Downloaded number_prefix v0.3.0
  Downloaded ident_case v1.0.1
  Downloaded thiserror v1.0.41
  Downloaded num_cpus v1.16.0
  Downloaded unicode-width v0.1.10
  Downloaded paste v1.0.11
  Downloaded rand_core v0.6.4
  Downloaded log v0.4.19
  Downloaded crossbeam-utils v0.8.16
  Downloaded monostate v0.1.8
  Downloaded onig v6.4.0
  Downloaded derive_builder_macro v0.12.0
  Downloaded derive_builder_core v0.12.0
  Downloaded crossbeam-deque v0.8.3
  Downloaded getrandom v0.2.10
  Downloaded unicode-ident v1.0.6
  Downloaded ryu v1.0.14
  Downloaded proc-macro2 v1.0.63
  Downloaded smallvec v1.11.0
  Downloaded darling v0.14.4
  Downloaded indicatif v0.15.0
  Downloaded console v0.15.7
  Downloaded once_cell v1.18.0
  Downloaded derive_builder v0.12.0
  Downloaded crossbeam-epoch v0.9.15
  Downloaded cc v1.0.79
  Downloaded darling_core v0.14.4
  Downloaded serde_derive v1.0.166
  Downloaded serde v1.0.166
  Downloaded rayon-core v1.11.0
  Downloaded minimal-lexical v0.2.1
  Downloaded aho-corasick v0.7.20
  Downloaded base64 v0.13.1
  Downloaded unicode_categories v0.1.1
  Downloaded rand v0.8.5
  Downloaded crossbeam-channel v0.5.8
  Downloaded memchr v2.5.0
  Downloaded itertools v0.9.0
  Downloaded itertools v0.8.2
  Downloaded unicode-normalization-alignments v0.1.12
  Downloaded unicode-segmentation v1.10.1
  Downloaded nom v7.1.3
  Downloaded rayon v1.7.0
  Downloaded serde_json v1.0.100
  Downloaded tokenizers v0.13.3
  Downloaded aho-corasick v1.0.2
  Downloaded esaxx-rs v0.1.8
  Downloaded syn v2.0.23
  Downloaded regex v1.9.0
  Downloaded syn v1.0.107
  Downloaded regex-syntax v0.6.29
  Downloaded regex-syntax v0.7.3
  Downloaded regex-automata v0.3.0
  Downloaded onig_sys v69.8.1
  Downloaded spm_precompiled v0.1.4
  Downloaded libc v0.2.147
  Downloaded libR-sys v0.5.0
   Compiling proc-macro2 v1.0.63
   Compiling quote v1.0.29
   Compiling unicode-ident v1.0.6
   Compiling libc v0.2.147
   Compiling cfg-if v1.0.0
   Compiling syn v1.0.107
   Compiling autocfg v1.1.0
   Compiling memchr v2.5.0
   Compiling crossbeam-utils v0.8.16
   Compiling cc v1.0.79
   Compiling ident_case v1.0.1
   Compiling serde v1.0.166
   Compiling strsim v0.10.0
   Compiling fnv v1.0.7
   Compiling scopeguard v1.1.0
   Compiling libR-sys v0.5.0
   Compiling either v1.8.1
   Compiling rayon-core v1.11.0
   Compiling paste v1.0.11
   Compiling pkg-config v0.3.27
   Compiling regex-syntax v0.7.3
   Compiling lazy_static v1.4.0
   Compiling serde_json v1.0.100
   Compiling unicode-width v0.1.10
   Compiling ppv-lite86 v0.2.17
   Compiling minimal-lexical v0.2.1
   Compiling extendr-engine v0.4.0 (https://github.com/extendr/extendr.git#836b0cc0)
   Compiling thiserror v1.0.41
   Compiling smallvec v1.11.0
   Compiling extendr-api v0.4.0 (https://github.com/extendr/extendr.git#836b0cc0)
   Compiling macro_rules_attribute-proc_macro v0.1.3
   Compiling once_cell v1.18.0
   Compiling itertools v0.8.2
   Compiling bitflags v1.3.2
   Compiling itoa v1.0.8
   Compiling unicode-segmentation v1.10.1
   Compiling ryu v1.0.14
   Compiling number_prefix v0.3.0
   Compiling base64 v0.13.1
   Compiling itertools v0.9.0
error: failed to run custom build command for `libR-sys v0.5.0`

Caused by:
  process didn't exit successfully: `/data/gannet/ripley/R/packages/tests-MKL/tok/src/./rust/target/release/build/libR-sys-ed5ac96bdc0fff0e/build-script-build` (exit status: 101)
  --- stdout
  cargo:rustc-env=R_HOME=/data/gannet/ripley/R/MKL
  cargo:r_home=/data/gannet/ripley/R/MKL

  --- stderr
  thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 2, kind: NotFound, message: "No such file or directory" }', /data/gannet/ripley/R/packages/tests-MKL/tok/src/.cargo/registry/src/github.com-1ecc6299db9ec823/libR-sys-0.5.0/build.rs:637:40
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
make[2]: *** [Makevars:16: rust/target/release/libtok.a] Error 101
make[2]: Target 'all' not remade because of errors.
make[2]: Leaving directory '/data/gannet/ripley/R/packages/tests-MKL/tok/src'
ERROR: compilation failed for package ‘tok’
* removing ‘/data/gannet/ripley/R/packages/tests-MKL/tok.Rcheck/tok’
eitsupi commented 1 year ago

Since prqlr has not received any warnings, perhaps the easiest fix is to use extendr 0.4.0. Is there any reason to use an unreleased version of extendr?

dfalbel commented 1 year ago

Thanks @eitsupi ! I actually didn't notice I was using an unreleased version of extendr :) I'll see if tok works after downgrading to extendr 0.4.0.