search

pola-rs / r-polars

Bring polars to R
https://pola-rs.github.io/r-polars/
Other
405 stars 35 forks source link

Binaries for MacOS - ARM...? #371

Closed coforfe closed 8 months ago

coforfe commented 9 months ago

Hello,

I am trying to install polars in a MacOs M2 Max machine and I get this error message:

> install.packages("polars", repos = "https://rpolars.r-universe.dev")
Package which is only available in source form, and may need compilation of
  C/C++/Fortran: ‘polars’
Do you want to attempt to install these from sources? (Yes/no/cancel) Yes
installing the source package ‘polars’

trying URL 'https://rpolars.r-universe.dev/src/contrib/polars_0.8.0.9000.tar.gz'
Content type 'application/x-gzip' length 1344988 bytes (1.3 MB)
==================================================
downloaded 1.3 MB

* installing *source* package ‘polars’ ...
** using staged installation
** libs
using C compiler: ‘gcc (GCC) 8.2.0’
using SDK: ‘’
rm -Rf polars.so ./rust/target/release/libr_polars.a entrypoint.o
/usr/local/gfortran/bin/gcc -fopenmp -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include    -fPIC  -g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe -c entrypoint.c -o entrypoint.o
if [ "true" != "true" ]; then \
        export CARGO_HOME=/private/var/folders/j9/m6xn7ss56bzb7x46wbc56yfm0000gn/T/Rtmp7s4OBK/R.INSTALL5495732c4e1c/polars/src/.cargo; \
    fi && \
    export PATH="/opt/local/bin:/opt/local/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/Library/Apple/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/Users/carlosortega/Applications/quarto/bin:/Library/TeX/texbin:/usr/texbin:/Applications/RStudio.app/Contents/Resources/app/quarto/bin:/Applications/RStudio.app/Contents/Resources/app/bin/postback:/Users/carlosortega/.cargo/bin" && \
    if [ "" == "true" ]; then \
        cargo build --lib --profile release --manifest-path="./rust/Cargo.toml" --features "full_features"; \
    else \
        cargo build --lib --profile release --manifest-path="./rust/Cargo.toml"; \
    fi
/bin/sh: cargo: command not found
make: *** [rust/target/release/libr_polars.a] Error 127
ERROR: compilation failed for package ‘polars’
* removing ‘/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/polars’
Warning in install.packages :
  installation of package ‘polars’ had non-zero exit status

The downloaded source packages are in
    ‘/private/var/folders/j9/m6xn7ss56bzb7x46wbc56yfm0000gn/T/RtmpcOH6iO/downloaded_packages’

Before this, I downloaded and installed rust which is already available in my machine here:

/opt/homebrew/bin/cargo

But it seems that the install is not recognizing it.

Any hint about what to check/modify would be very much appreciated.

Thanks in advance, Carlos.

eitsupi commented 9 months ago

It seems /opt/homebrew/bin/ is not on the PATH. https://docs.brew.sh/FAQ#my-mac-apps-dont-find-homebrew-utilities

sorhawell commented 9 months ago

With M2 you implicitly need to install from source. Did you install rustc?(oups missed that line sorry :/) This may be of help: https://github.com/pola-rs/r-polars#build-from-source https://github.com/pola-rs/r-polars#system-dependencies

Unfortunately R does not out-of-the-box* support cross OS/arch building of R packages. Both our binary releases and R-universe binary releases relies on building R packages on the native OS/arch with the available instances of github. Mac M2 machines are not freely available yet for open source projects.

*: We have previously ventured into performing rust cross-compilation to provide Mac M2 precompiled object files. The mac user still needs to have Xcode and make installed, but not rust. The R package is still built on mac users M2 machine, but it is quite faster to download precompiled object-file (~100Mb) and just link them (~1-2min depending on bandwidth). This can be incorporated into our makevars file. However, I dropped such cross-compilation for the time being because it was cumbersome (to me, but surely possible) to define a fully automatic CI/CD workflow. If anyone want to revive this, I can point to previous releases and sketch out some proposed changes.

coforfe commented 8 months ago

Thanks for your help!.

I installed again rustc and when running hellorust within R I get this error.

> remotes::install_github("r-rust/hellorust")
Using github PAT from envvar GITHUB_PAT
Downloading GitHub repo r-rust/hellorust@HEAD
── R CMD build ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────
✔  checking for file ‘/private/var/folders/j9/m6xn7ss56bzb7x46wbc56yfm0000gn/T/RtmpKE9f0I/remotes4f5622443dc5/r-rust-hellorust-8e5e619/DESCRIPTION’ ...
─  preparing ‘hellorust’:
✔  checking DESCRIPTION meta-information ...
─  cleaning src
─  running ‘cleanup’
─  checking for LF line-endings in source and make files and shell scripts (18.6s)
─  checking for empty or unneeded directories
─  building ‘hellorust_1.1.1.tar.gz’

* installing *source* package ‘hellorust’ ...
** using staged installation
cargo 1.72.0 (103a7ff2e 2023-08-15)
** libs
using C compiler: ‘gcc (GCC) 8.2.0’
using SDK: ‘’
/usr/local/gfortran/bin/gcc -fopenmp -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include   -pthread -fPIC  -g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe -c wrapper.c -o wrapper.o
In file included from /Library/Frameworks/R.framework/Resources/include/Rinternals.h:46,
                 from wrapper.c:3:
/Library/Frameworks/R.framework/Resources/include/R_ext/Complex.h:80:6: warning: ISO C99 doesn’t support unnamed structs/unions [-Wpedantic]
     };
      ^
if [ -f myrustlib/vendor.tar.xz ]; then tar xf myrustlib/vendor.tar.xz && mkdir -p /private/var/folders/j9/m6xn7ss56bzb7x46wbc56yfm0000gn/T/Rtmpsozqv5/R.INSTALL65cd7633dcb0/hellorust/src/.cargo && cp myrustlib/vendor-config.toml /private/var/folders/j9/m6xn7ss56bzb7x46wbc56yfm0000gn/T/Rtmpsozqv5/R.INSTALL65cd7633dcb0/hellorust/src/.cargo/config.toml; fi
PATH="/opt/local/bin:/opt/local/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/Library/Apple/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/Users/carlosortega/Applications/quarto/bin:/Library/TeX/texbin:/usr/texbin:/Applications/RStudio.app/Contents/Resources/app/quarto/bin:/Applications/RStudio.app/Contents/Resources/app/bin/postback:/Users/carlosortega/.cargo/bin" cargo build -j 2 --release --manifest-path=myrustlib/Cargo.toml
   Compiling libc v0.2.147
   Compiling rand v0.4.6
   Compiling myrustlib v0.1.0 (/private/var/folders/j9/m6xn7ss56bzb7x46wbc56yfm0000gn/T/Rtmpsozqv5/R.INSTALL65cd7633dcb0/hellorust/src/myrustlib)
    Finished release [optimized] target(s) in 1.07s
rm -Rf /private/var/folders/j9/m6xn7ss56bzb7x46wbc56yfm0000gn/T/Rtmpsozqv5/R.INSTALL65cd7633dcb0/hellorust/src/.cargo vendor || true # CRAN wants us to remove "detritus"
rm -Rf myrustlib/target/release/build || true
/usr/local/gfortran/bin/gcc -fopenmp -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/opt/gettext/lib -L/usr/local/opt/llvm/lib -Wl,-rpath,/usr/local/opt/llvm/lib -o hellorust.so wrapper.o -Lmyrustlib/target/release -lmyrustlib -lresolv -pthread -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
ld: warning: ignoring file myrustlib/target/release/libmyrustlib.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
ld: warning: ignoring file /Library/Frameworks/R.framework/R, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
ld: warning: dylib (/usr/local/opt/llvm/lib/libunwind.dylib) was built for newer macOS version (13.0) than being linked (10.18)
ld: warning: dylib (/usr/local/opt/llvm/lib/libunwind.dylib) was built for newer macOS version (13.0) than being linked (10.18)
installing to /Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/00LOCK-hellorust/00new/hellorust/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded from temporary location
Error: package or namespace load failed for ‘hellorust’ in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/00LOCK-hellorust/00new/hellorust/libs/hellorust.so':
  dlopen(/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/00LOCK-hellorust/00new/hellorust/libs/hellorust.so, 0x0006): tried: '/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/00LOCK-hellorust/00new/hellorust/libs/hellorust.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')), '/System/Volumes/Preboot/Cryptexes/OS/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/00LOCK-hellorust/00new/hellorust/libs/hellorust.so' (no such file), '/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/00LOCK-hellorust/00new/hellorust/libs/hellorust.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64'))
Error: loading failed
Execution halted
ERROR: loading failed
* removing ‘/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/hellorust’
Warning message:
In i.p(...) :
  installation of package ‘/var/folders/j9/m6xn7ss56bzb7x46wbc56yfm0000gn/T//RtmpKE9f0I/file4f562e336216/hellorust_1.1.1.tar.gz’ had non-zero exit status

As you see, it recognizes that cargois available, but it continues saying that sees my machine as a x86_64 machine... :-(.

Thanks again for all your help!. Carlos.

eitsupi commented 8 months ago

@coforfe Can you explain the exact versions of cargo and rustc and how you installed them? (However, I think these are not problems with this repository and we may not be able to solve them, so I recommend asking questions on StackOverflow etc.)

coforfe commented 8 months ago

OK. Thanks!.

Yes, these are the versions I have:

(base) carlosortega@MacBook-Pro ~ % cargo --version
cargo 1.72.0 (103a7ff2e 2023-08-15)
(base) carlosortega@MacBook-Pro ~ % rustc --version
rustc 1.72.0 (5680fa18f 2023-08-23)

Thanks again, Carlos.

sorhawell commented 8 months ago

GCC 8 is older and it is not clang, did you install xcode as in guide?

Personally I prefer helloextendr as minimal example repo to check compilation, I'm not too familiar with hellorust.

https://github.com/extendr/helloextendr

sorhawell commented 8 months ago

how the install isch should look like

✔ Saving changes in the open files.
ℹ Generating extendr wrapper functions for package: polars.
ℹ Re-compiling polars (debug build)
── R CMD INSTALL ────────────────────────────────────────────────────────────────────────────────────────────────────────────────
─  installing *source* package ‘polars’ ... (378ms)
   ** using staged installation
   ** libs
   using C compiler: ‘Apple clang version 14.0.0 (clang-1400.0.29.202)’
   using SDK: ‘’
   rm -Rf polars.so ./rust/target/release/libr_polars.a entrypoint.o
   clang -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include    -fPIC  -g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe -UNDEBUG -Wall -pedantic -g -O0 -fdiagnostics-color=always -c entrypoint.c -o entrypoint.o
   if [ "true" != "true" ]; then \
        export CARGO_HOME=/Users/sorenwelling/Documents/projs/r-polars/src/.cargo; \
    fi && \
    export PATH="/Users/sorenwelling/Documents/projs/r-polars/.venv/bin:/usr/local/opt/python@3.10/libexec/bin:/usr/local/opt/python@3.9/libexec/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Users/sorenwelling/.cargo/bin/:/Library/TeX/texbin:/opt/X11/bin:/Library/Apple/usr/bin:/Users/sorenwelling/.cargo/bin:/usr/ucb:/Applications/Docker.app/Contents/Resources/bin/:/Users/sorenwelling/Applications/quarto/bin:/usr/texbin:/Applications/RStudio.app/Contents/Resources/app/quarto/bin:/Applications/RStudio.app/Contents/Resources/app/bin/postback:/Users/sorenwelling/.cargo/bin" && \
    if [ "true" == "true" ]; then \
        cargo build --lib --profile release --manifest-path="./rust/Cargo.toml" --features "full_features"; \
    else \
        cargo build --lib --profile release --manifest-path="./rust/Cargo.toml"; \
    fi
      Compiling r-polars v0.1.0 (/Users/sorenwelling/Documents/projs/r-polars/src/rust)
coforfe commented 8 months ago

Thanks Søren,

Finally I could install it.

This is what I did:

CC = /opt/homebrew/bin/gcc-13
CXX = /opt/homebrew/bin/g++-13
FC = /opt/homebrew/bin/gfortran-13
F77 = /opt/homebrew/bin/gfortran-13
FLIBS = -L//opt/homebrew/lib/gcc/ -lgfortran -lm

It seemed that the Makevars I was using, was pointing out to x86_64 compilers.

And with this change, this is what I got during the installation process:

> install.packages("polars", repos = "https://rpolars.r-universe.dev")
Package which is only available in source form, and may need compilation of C/C++/Fortran: ‘polars’
Do you want to attempt to install these from sources? (Yes/no/cancel) Yes
installing the source package ‘polars’

trying URL 'https://rpolars.r-universe.dev/src/contrib/polars_0.8.0.9000.tar.gz'
Content type 'application/x-gzip' length 1346312 bytes (1.3 MB)
==================================================
downloaded 1.3 MB

* installing *source* package ‘polars’ ...
** using staged installation
** libs
using C compiler: ‘gcc-13 (Homebrew GCC 13.2.0) 13.2.0’
using SDK: ‘’
rm -Rf polars.so ./rust/target/release/libr_polars.a entrypoint.o
/opt/homebrew/bin/gcc-13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/R/arm64/include    -fPIC  -falign-functions=64 -Wall -g -O2  -c entrypoint.c -o entrypoint.o
if [ "true" != "true" ]; then \
        export CARGO_HOME=/private/var/folders/j9/m6xn7ss56bzb7x46wbc56yfm0000gn/T/Rtmpdf0vk5/R.INSTALL9bf718c2d1f2/polars/src/.cargo; \
    fi && \
    export PATH="/opt/local/bin:/opt/local/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/Library/Apple/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/Users/carlosortega/.cargo/bin:/usr/ucb:/Users/carlosortega/Applications/quarto/bin:/Library/TeX/texbin:/usr/texbin:/Applications/RStudio.app/Contents/Resources/app/quarto/bin:/Applications/RStudio.app/Contents/Resources/app/bin/postback:/Users/carlosortega/.cargo/bin" && \
    if [ "" == "true" ]; then \
        cargo build --lib --profile release --manifest-path="./rust/Cargo.toml" --features "full_features"; \
    else \
        cargo build --lib --profile release --manifest-path="./rust/Cargo.toml"; \
    fi
   Compiling autocfg v1.1.0
   Compiling libc v0.2.147
   Compiling proc-macro2 v1.0.66
   Compiling unicode-ident v1.0.11
   Compiling cfg-if v1.0.0
   Compiling version_check v0.9.4
   Compiling serde v1.0.185
   Compiling scopeguard v1.2.0
   Compiling libm v0.2.7
   Compiling crossbeam-utils v0.8.16
   Compiling memchr v2.5.0
   Compiling once_cell v1.18.0
   Compiling futures-core v0.3.28
   Compiling rayon-core v1.11.0
   Compiling static_assertions v1.1.0
   Compiling siphasher v0.3.10
   Compiling num-traits v0.2.16
   Compiling memoffset v0.9.0
   Compiling ahash v0.8.3
   Compiling crossbeam-epoch v0.9.15
   Compiling syn v1.0.109
   Compiling crossbeam-channel v0.5.8
   Compiling phf_shared v0.11.2
   Compiling slab v0.4.8
   Compiling quote v1.0.33
   Compiling lexical-util v0.8.5
   Compiling pin-project-lite v0.2.12
   Compiling syn v2.0.29
   Compiling pkg-config v0.3.27
   Compiling regex-syntax v0.7.4
   Compiling futures-task v0.3.28
   Compiling jobserver v0.1.26
   Compiling futures-channel v0.3.28
   Compiling futures-sink v0.3.28
   Compiling cc v1.0.83
   Compiling rand_core v0.6.4
   Compiling either v1.9.0
   Compiling rand v0.8.5
   Compiling getrandom v0.2.10
   Compiling num_cpus v1.16.0
   Compiling regex-automata v0.3.6
   Compiling crossbeam-deque v0.8.3
   Compiling zstd-sys v2.0.8+zstd.1.5.5
   Compiling allocator-api2 v0.2.16
   Compiling crc32fast v1.3.2
   Compiling futures-util v0.3.28
   Compiling target-features v0.1.4
   Compiling phf_generator v0.11.2
   Compiling cmake v0.1.50
   Compiling itoa v1.0.9
   Compiling ryu v1.0.15
   Compiling rayon v1.7.0
   Compiling semver v1.0.18
   Compiling pin-utils v0.1.0
   Compiling serde_json v1.0.105
   Compiling equivalent v1.0.1
   Compiling futures-io v0.3.28
   Compiling regex v1.9.3
   Compiling core-foundation-sys v0.8.4
   Compiling libz-ng-sys v1.1.12
   Compiling parse-zoneinfo v0.3.0
   Compiling phf_codegen v0.11.2
   Compiling lz4-sys v1.9.4
   Compiling lexical-write-integer v0.8.5
   Compiling lexical-parse-integer v0.8.6
   Compiling phf v0.11.2
   Compiling snap v1.1.0
   Compiling zstd-safe v6.0.6
   Compiling alloc-no-stdlib v2.0.4
   Compiling async-trait v0.1.73
   Compiling alloc-stdlib v0.2.2
   Compiling chrono-tz-build v0.2.0
   Compiling lexical-parse-float v0.8.5
   Compiling lexical-write-float v0.8.5
   Compiling iana-time-zone v0.1.57
   Compiling fallible-streaming-iterator v0.1.9
   Compiling adler v1.0.2
   Compiling rle-decode-fast v1.0.3
   Compiling hashbrown v0.14.0
   Compiling libflate_lz77 v1.2.0
   Compiling lexical-core v0.8.5
   Compiling miniz_oxide v0.7.1
   Compiling chrono-tz v0.8.3
   Compiling rustc_version v0.4.0
   Compiling brotli-decompressor v2.3.4
   Compiling serde_derive v1.0.185
   Compiling futures-macro v0.3.28
   Compiling async-stream-impl v0.3.5
   Compiling aho-corasick v1.0.4
   Compiling crc-catalog v1.1.1
   Compiling adler32 v1.2.0
   Compiling array-init-cursor v0.2.0
   Compiling planus v0.3.1
   Compiling libflate v1.4.0
   Compiling crc v2.1.0
   Compiling brotli v3.3.4
   Compiling async-stream v0.3.5
   Compiling arrow2 v0.17.4
   Compiling multiversion-macros v0.7.3
   Compiling bytemuck_derive v1.4.1
   Compiling streaming-decompression v0.1.2
   Compiling lock_api v0.4.10
   Compiling seq-macro v0.3.5
   Compiling simdutf8 v0.1.4
   Compiling thiserror v1.0.47
   Compiling log v0.4.20
   Compiling futures-executor v0.3.28
   Compiling multiversion v0.7.3
   Compiling futures v0.3.28
   Compiling parquet-format-safe v0.2.4
   Compiling bytemuck v1.13.1
   Compiling thiserror-impl v1.0.47
   Compiling streaming-iterator v0.1.9
   Compiling strength_reduce v0.2.4
   Compiling regex-syntax v0.6.29
   Compiling dyn-clone v1.0.13
   Compiling foreign_vec v0.1.0
   Compiling ethnum v1.3.2
   Compiling base64 v0.21.2
   Compiling hash_hasher v2.0.3
   Compiling smartstring v1.0.1
   Compiling signal-hook v0.3.17
   Compiling rustversion v1.0.14
   Compiling parking_lot_core v0.9.8
   Compiling indexmap v2.0.0
   Compiling chrono v0.4.26
   Compiling arrow-format v0.8.1
   Compiling mio v0.8.8
   Compiling signal-hook-registry v1.4.1
   Compiling avro-schema v0.3.0
   Compiling polars-utils v0.32.0 (https://github.com/pola-rs/polars.git?rev=ec0c91f93fcd1ac355c667d6c3c3f30b257ea0a6#ec0c91f9)
   Compiling ppv-lite86 v0.2.17
   Compiling smallvec v1.11.0
   Compiling rand_chacha v0.3.1
   Compiling bitflags v2.4.0
   Compiling sysinfo v0.29.8
   Compiling hashbrown v0.13.2
   Compiling polars-arrow v0.32.0 (https://github.com/pola-rs/polars.git?rev=ec0c91f93fcd1ac355c667d6c3c3f30b257ea0a6#ec0c91f9)
   Compiling matrixmultiply v0.3.7
   Compiling num-integer v0.1.45
   Compiling heck v0.4.1
   Compiling halfbrown v0.2.4
   Compiling signal-hook-mio v0.2.3
   Compiling parking_lot v0.12.1
   Compiling float-cmp v0.9.0
   Compiling atoi v2.0.0
   Compiling rawpointer v0.2.1
   Compiling bitflags v1.3.2
   Compiling crossterm v0.26.1
   Compiling value-trait v0.6.1
   Compiling strum_macros v0.24.3
   Compiling num-complex v0.4.4
   Compiling polars-core v0.32.0 (https://github.com/pola-rs/polars.git?rev=ec0c91f93fcd1ac355c667d6c3c3f30b257ea0a6#ec0c91f9)
   Compiling unicode-width v0.1.10
   Compiling strum v0.24.1
   Compiling simd-json v0.10.6
   Compiling rand_distr v0.4.3
   Compiling ndarray v0.15.6
   Compiling polars-ops v0.32.0 (https://github.com/pola-rs/polars.git?rev=ec0c91f93fcd1ac355c667d6c3c3f30b257ea0a6#ec0c91f9)
   Compiling itoap v1.0.1
   Compiling xxhash-rust v0.8.6
   Compiling jsonpath_lib v0.3.0 (https://github.com/ritchie46/jsonpath?branch=improve_compiled#24eaf0b4)
   Compiling argminmax v0.6.1
   Compiling hex v0.4.3
   Compiling now v0.1.3
   Compiling socket2 v0.5.3
   Compiling lexical v6.1.1
   Compiling memmap2 v0.7.1
   Compiling polars-plan v0.32.0 (https://github.com/pola-rs/polars.git?rev=ec0c91f93fcd1ac355c667d6c3c3f30b257ea0a6#ec0c91f9)
   Compiling tokio v1.32.0
   Compiling crossbeam-queue v0.3.8
   Compiling fast-float v0.2.0
   Compiling comfy-table v7.0.1
   Compiling libR-sys v0.4.0
   Compiling home v0.5.5
   Compiling bytes v1.4.0
   Compiling strum_macros v0.25.2
   Compiling polars-pipe v0.32.0 (https://github.com/pola-rs/polars.git?rev=ec0c91f93fcd1ac355c667d6c3c3f30b257ea0a6#ec0c91f9)
   Compiling enum_dispatch v0.3.12
   Compiling polars-lazy v0.32.0 (https://github.com/pola-rs/polars.git?rev=ec0c91f93fcd1ac355c667d6c3c3f30b257ea0a6#ec0c91f9)
   Compiling rustix v0.38.8
   Compiling libmimalloc-sys v0.1.33
   Compiling errno v0.3.2
   Compiling extendr-engine v0.4.0 (https://github.com/rpolars/extendr?branch=pl0.7.0rc#ee8d3754)
   Compiling glob v0.3.1
   Compiling paste v1.0.14
   Compiling sqlparser v0.36.1
   Compiling polars v0.32.0 (https://github.com/pola-rs/polars.git?rev=ec0c91f93fcd1ac355c667d6c3c3f30b257ea0a6#ec0c91f9)
   Compiling fastrand v2.0.0
   Compiling extendr-api v0.4.0 (https://github.com/rpolars/extendr?branch=pl0.7.0rc#ee8d3754)
   Compiling lazy_static v1.4.0
   Compiling tempfile v3.8.0
   Compiling bincode v1.3.3
   Compiling spin v0.9.8
   Compiling extendr-macros v0.4.0 (https://github.com/rpolars/extendr?branch=pl0.7.0rc#ee8d3754)
   Compiling uuid v1.4.1
   Compiling nanorand v0.7.0
   Compiling fnv v1.0.7
   Compiling ipc-channel v0.17.0
   Compiling flume v0.11.0
   Compiling indenter v0.3.3
   Compiling state v0.6.0
   Compiling mimalloc v0.1.37
   Compiling lz4 v1.24.0
   Compiling flate2 v1.0.27
   Compiling zstd v0.12.4
   Compiling parquet2 v0.17.2
   Compiling polars-error v0.32.0 (https://github.com/pola-rs/polars.git?rev=ec0c91f93fcd1ac355c667d6c3c3f30b257ea0a6#ec0c91f9)
   Compiling polars-row v0.32.0 (https://github.com/pola-rs/polars.git?rev=ec0c91f93fcd1ac355c667d6c3c3f30b257ea0a6#ec0c91f9)
   Compiling polars-json v0.32.0 (https://github.com/pola-rs/polars.git?rev=ec0c91f93fcd1ac355c667d6c3c3f30b257ea0a6#ec0c91f9)
   Compiling polars-time v0.32.0 (https://github.com/pola-rs/polars.git?rev=ec0c91f93fcd1ac355c667d6c3c3f30b257ea0a6#ec0c91f9)
   Compiling polars-io v0.32.0 (https://github.com/pola-rs/polars.git?rev=ec0c91f93fcd1ac355c667d6c3c3f30b257ea0a6#ec0c91f9)
   Compiling polars-sql v0.32.0 (https://github.com/pola-rs/polars.git?rev=ec0c91f93fcd1ac355c667d6c3c3f30b257ea0a6#ec0c91f9)
   Compiling r-polars v0.1.0 (/private/var/folders/j9/m6xn7ss56bzb7x46wbc56yfm0000gn/T/Rtmpdf0vk5/R.INSTALL9bf718c2d1f2/polars/src/rust)
    Finished release [optimized] target(s) in 2m 13s
if [ "true" != "true" ]; then \
        rm -Rf /private/var/folders/j9/m6xn7ss56bzb7x46wbc56yfm0000gn/T/Rtmpdf0vk5/R.INSTALL9bf718c2d1f2/polars/src/.cargo && \
        rm -Rf ./rust/target/release/build; \
    fi
if [ -f "./rust/target/release/libr_polars.a" ]; then \
        echo "file is there: "; \
    else \
        echo "no, file is NOT there: "; \
        mkdir -p ./rust/target/release ; \
        echo "trying to symlink in "./rust/target/release/libr_polars.a""; \
        ln -s "./rust/target/release/libr_polars.a" ./rust/target/release/libr_polars.a ; \
    fi
file is there: 
if [ "" == "true" ]; then \
        echo "cleanup!!" ; \
        mv ./rust/target/release/libr_polars.a ./rust/target/release/../temp_binary.a; \
        rm -rf ./rust/target/release; \
        mkdir ./rust/target/release; \
        mv ./rust/target/release/../temp_binary.a ./rust/target/release/libr_polars.a; \
        rm -rf ./src/.cargo; \
    else \
        echo "hands off!!" ; \
    fi
hands off!!
/opt/homebrew/bin/gcc-13 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/arm64/lib -o polars.so entrypoint.o -L./rust/target/release -lr_polars -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
installing to /Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library/00LOCK-polars/00new/polars/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
** checking absolute paths in shared objects and dynamic libraries
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (polars)

The downloaded source packages are in
    ‘/private/var/folders/j9/m6xn7ss56bzb7x46wbc56yfm0000gn/T/RtmpwaD9tr/downloaded_packages’

Thanks again for your help. Carlos.

sorhawell commented 8 months ago

I have not tried compiling much without Xcode. Sales pitch ^^ : When installing Xcode, it often also just adds to PATH correctly so it just works. Xcode is often the recommended toolchain for R mac.

I guess you have experienced a PATH issue. It seems R found an old GCC8 and by hardcoding the path to your desired homebrew compiler, it works on your machine. I'm very happy for that :) If it is a PATH issue, you may benefit from adding /opt/homebrew/bin/ to your PATH.

We have had a handful who say the current Makevars does work when using xcode, but it is not something, I can confirm myself with a arm64 mac. So we don't mind some extra tales to refer back to :)

On a side note I'm very impressed your machine can compile the rust part in 2 minutes !! That takes 15min on my 2016 4 core macbook. Thank goodness for caching.

coforfe commented 8 months ago

Well, the issue is that I had Xcode installed:

(base) carlosortega@MacBook-Pro ~ % xcode-select --install
xcode-select: error: command line tools are already installed, use "Software Update" in System Settings to install updates

Well, I had before another Mac (x86_64) and even with Xcode installed too, I always have compiled some packages with particular Makevars, in particular one package data.table requires very special Makevars in order to make it use all cores available (OpenMP). With Xcode this was not possible.

Yes, it is a M2 Max machine...12 cores, and GPU with 30 cores and Neural Engine 16 cores...

Thanks again!

sorhawell commented 8 months ago

Some progress on cross-compilation here #4

sorhawell commented 8 months ago

https://community.rstudio.com/t/specifying-compiler-in-rstudio-on-mac/77912