walkerke / walkerke.github.io

Walker Data website
http://walkerke.github.io
19 stars 4 forks source link

overture building blog post open_dataset url not working? #10

Open zackarno opened 1 month ago

zackarno commented 1 month ago

Hi Kyle, thanks for you great blogpost on accessing data from overture.

I was following along with it and noticed the link in this line doesn't seem to work

https://github.com/walkerke/walkerke.github.io/blob/bc628b1fa7d0969a279f27252f71be311568dae1/posts/overture-buildings/index.qmd#L54

Below is a reprex:

library(arrow)
#> 
#> Attaching package: 'arrow'
#> The following object is masked from 'package:utils':
#> 
#>     timestamp
library(tidyverse)
library(aws.s3)
arrow::arrow_with_s3()
#> [1] TRUE

# from blog post:
buildings <- open_dataset('s3://overturemaps-us-west-2/release/2024-05-16-beta.0/theme=buildings?region=us-west-2')
#> Error: IOError: When getting information for key 'release/2024-05-16-beta.0/theme=buildings' in bucket 'overturemaps-us-west-2': AWS Error ACCESS_DENIED during HeadObject operation: No response body.

Created on 2024-06-07 with reprex v2.1.0

walkerke commented 1 month ago

I can't reproduce this error. When you run arrow::arrow_info(), what do you see?

zackarno commented 1 month ago
buildings <- open_dataset('s3://overturemaps-us-west-2/release/2024-05-16-beta.0/theme=buildings?region=us-west-2')
#> Error: IOError: When getting information for key 'release/2024-05-16-beta.0/theme=buildings' in bucket 'overturemaps-us-west-2': AWS Error ACCESS_DENIED during HeadObject operation: No response body.

arrow::arrow_info()
#> Arrow package version: 14.0.0.2
#> 
#> Capabilities:
#>                
#> acero      TRUE
#> dataset    TRUE
#> substrait FALSE
#> parquet    TRUE
#> json       TRUE
#> s3         TRUE
#> gcs        TRUE
#> utf8proc   TRUE
#> re2        TRUE
#> snappy     TRUE
#> gzip       TRUE
#> brotli     TRUE
#> zstd       TRUE
#> lz4        TRUE
#> lz4_frame  TRUE
#> lzo       FALSE
#> bz2        TRUE
#> jemalloc   TRUE
#> mimalloc   TRUE
#> 
#> Memory:
#>                   
#> Allocator mimalloc
#> Current    0 bytes
#> Max        0 bytes
#> 
#> Runtime:
#>                         
#> SIMD Level          none
#> Detected SIMD Level none
#> 
#> Build:
#>                                                              
#> C++ Library Version                                    14.0.0
#> C++ Compiler                                       AppleClang
#> C++ Compiler Version                          15.0.0.15000040
#> Git ID               2dcee3f82c6cf54b53a64729fd81840efa583244
sessionInfo()
#> R version 4.3.3 (2024-02-29)
#> Platform: aarch64-apple-darwin20 (64-bit)
#> Running under: macOS Sonoma 14.5
#> 
#> Matrix products: default
#> BLAS:   /Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/lib/libRblas.0.dylib 
#> LAPACK: /Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/lib/libRlapack.dylib;  LAPACK version 3.11.0
#> 
#> locale:
#> [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
#> 
#> time zone: America/New_York
#> tzcode source: internal
#> 
#> attached base packages:
#> [1] stats     graphics  grDevices utils     datasets  methods   base     
#> 
#> loaded via a namespace (and not attached):
#>  [1] styler_1.10.2     digest_0.6.34     fastmap_1.1.1     xfun_0.42        
#>  [5] magrittr_2.0.3    glue_1.7.0        R.utils_2.12.3    knitr_1.45       
#>  [9] htmltools_0.5.7   rmarkdown_2.25    lifecycle_1.0.4   cli_3.6.2        
#> [13] R.methodsS3_1.8.2 vctrs_0.6.5       reprex_2.0.2      withr_3.0.0      
#> [17] compiler_4.3.3    R.oo_1.25.0       R.cache_0.16.0    purrr_1.0.2      
#> [21] rstudioapi_0.15.0 tools_4.3.3       evaluate_0.23     yaml_2.3.8       
#> [25] rlang_1.1.3       fs_1.6.3

Created on 2024-06-08 with reprex v2.0.2