search

eco4cast / neon4cast

A helper R package for the neon4cast challenge
Other
7 stars 7 forks source link

error in noaa_stageX functions #16

Closed jmzobitz closed 1 month ago

jmzobitz commented 1 month ago

running weather <- noaa_stage1()

I get the following error message: Error in (function (anonymous, access_key, secret_key, session_token, : Cannot call fs___S3FileSystem__create(). See https://arrow.apache.org/docs/r/articles/install.html for help installing Arrow C++ libraries.

relevant R versions:

Searching the web for help wasn't useful - I don't know if this is related to my local install.

Thanks for any insight!

cboettig commented 1 month ago

@jmzobitz thanks for the report! Looks like your local install of arrow was built without support for S3. Did the instructions linked in the error message above (https://arrow.apache.org/docs/r/articles/install.html) provide any help?

Can you let us know the output of sessionInfo()?

jmzobitz commented 1 month ago

Unfortunately no. I am starting to suspect that this may be an issue with my computer (newer Macbook Air with Apple M2), so I understand that I might need to sort this out on my end.

  1. Here is the sessionInfo:

R version 4.4.0 (2024-04-24) Platform: aarch64-apple-darwin20 Running under: macOS Ventura 13.5.2

Matrix products: default BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib

LAPACK: /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/lib/libRlapack.dylib; LAPACK version 3.12.0

locale: [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

time zone: America/Chicago tzcode source: internal

attached base packages: [1] stats graphics grDevices utils datasets methods base

other attached packages: [1] neon4cast_0.1.0

loaded via a namespace (and not attached): [1] assertthat_0.2.1 utf8_1.2.4 R6_2.5.1 bit_4.0.5 [5] tidyselect_1.2.1 magrittr_2.0.3 glue_1.7.0 tibble_3.2.1 [9] pkgconfig_2.0.3 bit64_4.0.5 dplyr_1.1.4 generics_0.1.3 [13] lifecycle_1.0.4 cli_3.6.2 fansi_1.0.6 arrow_15.0.1 [17] vctrs_0.6.5 compiler_4.4.0 purrr_1.0.2 rstudioapi_0.16.0 [21] tools_4.4.0 pillar_1.9.0 rlang_1.1.3

  1. Based on the instructions in https://arrow.apache.org/docs/r/articles/install.html I ran this:

options( HTTPUserAgent = sprintf( "R/%s R (%s)", getRversion(), paste(getRversion(), R.version["platform"], R.version["arch"], R.version["os"]) ) )

install.packages("arrow", repos = " https://packagemanager.rstudio.com/all/__linux__/focal/latest")

But received the following message: pkg-config not found. OpenSSL not found *** Proceeding without libarrow (build not authorized) ------------------------- NOTE --------------------------- There was an issue building the Arrow C++ libraries. See https://arrow.apache.org/docs/r/articles/install.html

  1. I realized that I needed to install homebrew and install pkg-config and OpenSSL, which were successful in homebrew. However I've still received the same message when installing again.

As I said, this may be an issue on my end of the computer which I might need to figure out. I appreciate any insight.

John

On Tue, May 14, 2024 at 11:14 PM Carl Boettiger @.***> wrote:

@jmzobitz https://github.com/jmzobitz thanks for the report! Looks like your local install of arrow was built without support for S3. Did the instructions linked in the error message above ( https://arrow.apache.org/docs/r/articles/install.html) provide any help?

Can you let us know the output of sessionInfo()?

— Reply to this email directly, view it on GitHub https://github.com/eco4cast/neon4cast/issues/16#issuecomment-2111547517, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACPZNTRLNEOCT62GUWQ5MHTZCLOLJAVCNFSM6AAAAABHXCZH7WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJRGU2DONJRG4 . You are receiving this because you were mentioned.Message ID: @.***>

cboettig commented 1 month ago

@jmzobitz interesting! Support for Mac's arm based chips has been tricky historically, but CRAN should have Mac arm binaries available that should have support for S3 built in. However, sometimes things still go pear-shaped, especially so soon after the annual release of the new version of R last month.

Did you install arrow using install.packages() ? Can you try reinstalling with that? Note that if asked if you want to install from source (it will sometimes say something about a newer version being available from source), make sure you choose the answer "no". (This message is meaningless to many users, who figure the safe choice is "yes" -- poor UI design! building from source requires you have all the libraries installed).

Can you also just double-check that in your current setup, S3 is not present by confirming that the following returns FALSE: 

arrow::arrow_with_s3()
jmzobitz commented 1 month ago

Thanks! arrow::arrow_with_s3() was FALSE unfortunately, even after reinstalling from source. Unfortunately that didn't work.

I'll keep trying - the maddening part was that last summer at the ESA workshop installation worked beautifully, then my computer was updated by the University.

John M. Zobitz, PhD Pronouns: He, him, his

Professor of Mathematics and Data Science Data Science Program Director Environmental Studies Program Augsburg University 2211 Riverside Avenue, CB 93 Minneapolis, MN 55454

Hagfors Center 187 P: (612) 330-1068

On Wed, May 15, 2024 at 9:46 AM Carl Boettiger @.***> wrote:

@jmzobitz https://github.com/jmzobitz interesting! Support for Mac's arm based chips has been tricky historically, but CRAN should have Mac arm binaries available that should have support for S3 built in. However, sometimes things still go pear-shaped, especially so soon after the annual release of the new version of R last month.

Did you install arrow using install.packages() ? Can you try reinstalling with that? Note that if asked if you want to install from source (it will sometimes say something about a newer version being available from source), make sure you choose the answer "no". (This message is meaningless to many users, who figure the safe choice is "yes" -- poor UI design! building from source requires you have all the libraries installed).

Can you also just double-check that in your current setup, S3 is not present by confirming that the following returns FALSE:

arrow::arrow_with_s3()

— Reply to this email directly, view it on GitHub https://github.com/eco4cast/neon4cast/issues/16#issuecomment-2112747329, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACPZNTTIWAKFKS3COEERTF3ZCNYNBAVCNFSM6AAAAABHXCZH7WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJSG42DOMZSHE . You are receiving this because you were mentioned.Message ID: @.***>

cboettig commented 1 month ago

@jmzobitz right, sorry I didn't word that well. By 'answer "no"' to install from source I meant that you do want to install the prebuilt binary directly from CRAN, which should have S3 support built in.

I think the install from source is the reason you don't have s3 support. building arrow from source with support for all the options is rather complex, as detailed in the docs. https://arrow.apache.org/docs/r/articles/install.html#dependencies-for-s3-and-gcs-support. much easier to install the prebuilt binary.

jmzobitz commented 1 month ago

Finally found success! 🎉

Thanks for helping me (1) understand the error message (especially since my error message was outside of the package) and (2) pointing me to helpful resources. For posterity, I am going to record the steps that worked in case others have similar issues (working on a Macbook M2 chip). Happy to make additional edits if these aren't correct or delete if way off topic.

Homerew

Path correction

Homebrew installed some packages not in the main path, so I had to edit my ~/.zshrc file:

Install arrow from source in R:

Installation took a while, but was successful when done!

Thanks again, John