paws.storage::s3() gives error in vapply

[jimgar]

Hi there

Thanks for this amazing package. I use it a lot.

I have some functions that rely on paws.storage::s3()$get_object(). They have just started erroring out. I've found that the error seems to be coming from paws.storage::s3() itself.

Here's the error message:

> paws.storage::s3()
Error in vapply(sub_content, function(x) grepl("^[ ]+", x[1]), logical(1)) : 
  values must be length 1,
 but FUN(X[[1]]) result is length 0

And here's the traceback. I've omitted the first two steps because they are my own functions at the end of the chain

11.
vapply(sub_content, function(x) grepl("^[ ]+", x[1]), logical(1))
10.
which(!vapply(sub_content, function(x) grepl("^[ ]+", x[1]),
logical(1)))
9.
nested_ini_content(sub_content, found_nested_content)
8.
read_ini(config_path)
7.
check_config_file_sts_regional_endpoint(profile)
6.
.optional_config_parameter[[cfg_param]](profile)
5.
update_optional_config_parameter(cfgs, cfgs$credentials$profile)
4.
set_config(svc, config)
3.
paws.storage::s3()

As far as I know nothing on my system has changed. In response I've updated R, reinstalled all my packages, restarted my laptop. Permissions are all kept in ~/.aws/credentials and I haven't changed that either (AWS cli has no problem finding them).

Any ideas?

Session info:

> sessionInfo()
R version 4.3.1 (2023-06-16)
Platform: aarch64-apple-darwin20 (64-bit)
Running under: macOS Monterey 12.6

Matrix products: default
BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib 
LAPACK: /Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/lib/libRlapack.dylib;  LAPACK version 3.11.0

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

time zone: Europe/London
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
 [1] httr_1.4.7         compiler_4.3.1     R6_2.5.1           paws.storage_0.4.0 tools_4.3.1       
 [6] rstudioapi_0.15.0  curl_5.0.2         crayon_1.5.2       Rcpp_1.0.11        paws.common_0.6.1 

[jimgar]

Hi,

This is now resolved. A colleague stepped through the various functions with me and had the idea to look in the config file.

My config file looked like

[default]
region=eu-west-2

[profile profilename]

The non-default profile was being picked up in step 10 of the traceback above, and that was the cause of the issue. We deleted the extraneous profile and that resolved the problem.

Now it looks like:

[default]
region=eu-west-2

Because this other profile was in my config for many months, I think this could have been an error introduced with the latest release of paws (0.4.0). I wonder if this could be a problem for other users with multiple profiles in the future?

Thanks again for maintaining such a wonderful resource for us all.

[jimgar]

Just to note that if I give the non-default profile as

[default]
region=eu-west-2

[profile profilename]
region=eu-west-2

I.e., I have also provided a region, then the function does not error.

Additionally, while my colleague could not replicate this behaviour pre-0.4.0, he can after updating to 0.4.0. Therefore I am reopening as this seems like a bug.

[DyfanJones]

Hi @jimgar thanks for the detailed ticket. Yes the ini parsers has been update in paws.common v0.6.1. This is causing the bug. It looks like the scenario you have highlighted isn't currently in our unit tests https://github.com/paws-r/paws/blob/main/paws.common/tests/testthat/data_ini.

I will add this scenario for the next release of paws.common

[DyfanJones]

PR #681 now fixes issue of reading empty profiles from ini files. Feel free to test it out 😄

remotes::install_github("DyfanJones/paws/paws.common", ref = "bug_ini")

I will leave this open until released onto the CRAN.

[LeeMendelowitz]

I ran into this issue as well while calling paws::batch(). A workaround is to use paws.common version 0.6.0. If you use renv, this can be installed with renv::install("paws.common@0.6.0")

[DyfanJones]

Hi @LeeMendelowitz please try latest version from r-universe, as this fixes the bug. The dev version will be released onto the cran early next week :)

[DyfanJones]

Closing as paws.common 0.6.2 has been released to the cran