Open mattocci27 opened 2 years ago
That's odd, as I don't have an AWS_REGION
variable in my environment. However, based on the output here, you are not actually using the min.io server cache, because it is building rather than skipping the targets. So there may be some other reason that the right endpoint is not passed to targets
.
Sorry for the confusion. I tried it again and it's actually skipped building.
r$> targets::tar_make()
✔ skip target data
✔ skip target model
✔ skip pipeline: 0.153 seconds
I can also access to a new bucket (different name) in the min.io server but I still need to specify AWS_REGION
. I copied the minio version. The R environment is as in renv.lock. Someone may experience the same problem, so I just wanted to share. Thank you for the great tutorial!
> minio --version
minio version RELEASE.2022-07-17T15-43-14Z (commit-id=b6eb8dff649b0f46c12d24e89aa11254fb0132fa)
Runtime: go1.18.4 darwin/arm64
License: GNU AGPLv3 <https://www.gnu.org/licenses/agpl-3.0.html>
Copyright: 2015-0000 MinIO, Inc.
Hi there,
I ran into the same problem. Without an AWS_REGION set it fails. I get:
targets::tar_make() Error in get_region(cfgs$credentials$profile) : No region provided x Problem with the pipeline. i Show errors: tar_meta(fields = error, complete_only = TRUE) i Learn more: https://books.ropensci.org/targets/debugging.html Error: ! problem with the pipeline. Called from: signal_abort(cnd, .file)
If I set it, for example as suggested above to AWS_REGION="us-east-1"
it works fine. And it skips making the targets and finds them in minio. I can see it access the bucket when I Watch the bucket in the minio console.
Note when grepping for region we do find it in the meta file as region=NULL
and setting AWS_REGION="NULL"
works as well. It seems that paws::get_region(..) requires a region or will stop. It's unclear what the region is used for, but maybe it's adds it to the meta file when properly set?
Note: I used renv::restore(..) to get the project dependencies as documented - but I had to upgrade two packages (cpp11 [0.4.6], igraph [1.5.1]) to succeed on this. I'm on R 4.1 not 4.2 and on Windows.
Thanks for finding the workaround @mattocci27 btw! And of course thanks to you @noamross for the example repo.
I realized that even though I don't have AWS_REGION
set, I do have a region set in my .aws/credentials
file. Since targets uses the paws
package for AWS/S3 interaction, and that is AWS-specific, my guess is that the paws
specification is looking for region even though non-AWS services may not need it.
I originally got this error.
After adding a random region here (
AWS_REGION="us-east-1"
), the error has gone.https://github.com/noamross/targets-minio-versioning/blob/8c2dde713ba1e7d4b7b73c7174b8b4527418fead/_targets.R#L7-L10