mountainMath / cmhc

Wrapper for hack into CMHC data
Other
18 stars 5 forks source link

Inaccessible geographies #6

Closed bdbmax closed 1 year ago

bdbmax commented 1 year ago

Thanka a lot for this package, I look forward to using it to have much easier access to CMHC data!

While I can access data using cmhc::get_cmhc, I cannot access geographies using cmhc::get_cmhc_geography. It looks like the endpoint to the AWS bucket isn't right:

> cmhc::set_cmhc_cache_path("~/cmhc_cache", install = TRUE, overwrite = TRUE)
Your original .Renviron will be backed up and stored in your R HOME directory if needed.
Your cache path has been stored in your .Renviron and can be accessed by Sys.getenv("CMHC_CACHE_PATH").
[1] "~/cmhc_cache"
> cmhc::get_cmhc_geography(level = "ZONE")
Downloading geographies, this may take a minute...
List of 6
 $ Code     : chr "PermanentRedirect"
 $ Message  : chr "The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future "| __truncated__
 $ Endpoint : chr "mountainmath.s3.amazonaws.com"
 $ Bucket   : chr "mountainmath"
 $ RequestId: chr "DEVEV28Z0WV4E5FB"
 $ HostId   : chr "Wpn6zIAqge9ld7WR4zrY1cNmhGwV9tTfvVVw16Kxk6FVrY0AyYmfDrl+7xbYbTnbbeyZC7KTAG0="
 - attr(*, "headers")=List of 7
  ..$ x-amz-bucket-region: chr "ca-central-1"
  ..$ x-amz-request-id   : chr "DEVEV28Z0WV4E5FB"
  ..$ x-amz-id-2         : chr "Wpn6zIAqge9ld7WR4zrY1cNmhGwV9tTfvVVw16Kxk6FVrY0AyYmfDrl+7xbYbTnbbeyZC7KTAG0="
  ..$ content-type       : chr "application/xml"
  ..$ transfer-encoding  : chr "chunked"
  ..$ date               : chr "Thu, 06 Oct 2022 17:26:01 GMT"
  ..$ server             : chr "AmazonS3"
  ..- attr(*, "class")= chr [1:2] "insensitive" "list"
 - attr(*, "class")= chr "aws_error"
NULL
Error in parse_aws_s3_response(r, Sig, verbose = verbose) : 
  Moved Permanently (HTTP 301).
In addition: Warning message:
In dir.create(file.path(base_directory)) :
  'C:\Users\maxim\OneDrive - McGill University\Documents\cmhc_cache' already exists

Let me know if there's any more information from my end you'd need me to share.

Thank you!

mountainMath commented 1 year ago

Thanks for flagging, will look into this.

mountainMath commented 1 year ago

Ok, looks like I disabled public access to the S3 bucket at some point. I think I fixed that and it should work now. Please let me know if it continues to make problems.

Additionally I found a problem that on first run it would report the the download failed. But when running again everything will be fine. This is fixed on the v0.2.3 branch and will get updated on CRAN with the next update.

bdbmax commented 1 year ago

It seems to still not be working. It returns the exact same error (on both main and v0.2.3 branch)!

mountainMath commented 1 year ago

Ok, thanks for bearing with me. I think I figured out the root cause, I did not properly set the S3 bucket region for the data and my environment variables were masking that. It should now work on the v0.2.3 branch. Can you give it another try?

bdbmax commented 1 year ago

I confirm, it's fixed! Thanks for the quick answer, and thanks again for this package!