RedOakStrategic / redshiftTools

R Tools to use Amazon Redshift with R
Other
4 stars 0 forks source link

Error in if (attributes(resp) #8

Open tsydorenko opened 3 years ago

tsydorenko commented 3 years ago

Hey hey, I just installed your version via install_github("RedOakStrategic/redshiftTools") and unfortunately I'm getting this error while uploading data to S3

library(RPostgres)
library(redshiftTools)

con <- dbConnect(RPostgres::Postgres(), dbname="xxx",
                     host='xxx', port='5439',
                     user='xxx', password='xxx',sslmode='require')

test<- cbind.data.frame('value1','value2')
colnames(test)<- c("v1","v2")

rs_create_table(test,dbcon=con, 
                table_name='test_mb', 
                region="eu-west-1",bucket="xxx",
                access_key="xxx",
                secret_key="xxx",
)

Initiating Redshift table creation for table test_mb
Initiating Redshift table replacement for table test_mb
The provided data.frame has 1 rows and 2 columns
Getting number of slices from Redshift
1.18575755001899e-322 slices detected, will split into 1 files
Error in if (attributes(resp)$status_code %in% 404) { : 
  argument is of length zero

Do you have any idea why this is happening? Until now I was mostly using the older version from sicarul/redshiftTools and never had any problem with that.

JakeRuss commented 3 years ago

Hmm, I have not seen this error message yet. I will have to investigate. I use our version all the time without error. Does this message occur intermittently? @tsydorenko (ie, does the package ever work for you?)

It is totally possible I introduced a bug when I re-did some of the package internals.

tsydorenko commented 3 years ago

yes the package worked absolutely fine for me with this installation

devtools::install_github("r-dbi/RPostgres", force=T)
install.packages("aws.s3", repos = c("cloudyr" = "http://cloudyr.github.io/drat"))
devtools::install_github("sicarul/redshiftTools")

I thought that I can use your version because it is the most updated one and installed it via

install_github("RedOakStrategic/redshiftTools")

While debugging I noticed that I have an issue with the configuration for s3. Can you tell me how exactly you generate session token in for this


    config = list(
      credentials = list(
        creds = list(
          access_key_id     = Sys.getenv('AWS_ACCESS_KEY_ID'),
          secret_access_key = Sys.getenv('AWS_SECRET_ACCESS_KEY'),
          session_token     = Sys.getenv('AWS_SESSION_TOKEN')
        )
      ),
      region = Sys.getenv('AWS_DEFAULT_REGION')
    )
  )```
JakeRuss commented 3 years ago

We're not generating session tokens in this package --though maybe we should.

If you make a successful STS call with your AWS credentials and then set those in the global environment, then the functions should work for you.

What I meant by intermittent, was does our Red Oak version of the package ever work for you or does it always fail? (I suspect it always fails, because you need to set the environmental variables).