Closed danopia closed 3 years ago
I'm trying to point it to my local docker instance also, I'm running minio which exposes an S3 api.
Ok, so in a usecase where you'd like to give a specific URL when constructing the S3 client class, and path-based object access should then always be used (as opposed to the subdomain-style access that real S3 prefers)?
This is a bit different of a usecase than the original ones I listed, but also a bit simpler, so it should be pretty straightforward to add an escape hatch to enable this.
I understand that minio servers still have a region for signing purposes even though you won't be putting region into the URL. So the given base URL would effectively act like a global-endpoint override, except without forcing us-east-1.
I came here to write exactly this. I need to be able to connect to a different provider of AWS as well.
It's fine if there's a system automatically in place to point things to amazonaws.com, but I definitely need an override or I'm locked out.
I need to be able to connect to a different provider of AWS as well.
Hi @Scionax are you also looking at a localstack setup (special localhost URLs for every service) or an alternative cloud (proper HTTPS URLs, just different ones from commercial AWS)?
I can put in an escape hatch for the first case, but I could also do better detection of the second case if that lines up with your use case.
The nodejs api that amazon publishes works with minio in localhost by just using the options endpoint
and forcePathStyle
. You have to specify a region also but it doesn't matter which one and minio gives you the creds when its launched and is compatible in most other ways with the sdk as is.
I've opened a PR which adds a fixedEndpoint
option for the localstack/minio usecase, please feel free to give it a try now before I am ready to merge it and ship v0.5.0
.
import { ApiFactory } from "https://raw.githubusercontent.com/cloudydeno/deno-aws_api/fixed-endpoints/lib/client/mod.ts";
import { S3 } from "https://aws-api.deno.dev/v0.1/services/s3.ts"; // or whatever
const s3 = new S3(new ApiFactory({
fixedEndpoint: "http://localhost:9000",
}));
console.log("All buckets:", await s3.listBuckets()); // or whatever
I've done minimal testing with minio locally. (Might be cool for CI..)
Awesome, the only other thing is the forcePathStyle
will essentially put the bucket name into the path rather than the subdomain, does this automatically go into path style mode when using a fixed endpoint? If so then that makes sense.
I think if you can just do even a single request to a bucket based endpoint then you're good to go. I don't have the environment up at the moment to test it myself but if this works then its good
import { ApiFactory } from "https://raw.githubusercontent.com/cloudydeno/deno-aws_api/fixed-endpoints/lib/client/mod.ts";
import { S3 } from "https://aws-api.deno.dev/v0.1/services/s3.ts"; // or whatever
const s3 = new S3(new ApiFactory({
fixedEndpoint: "http://localhost:9000",
}));
await s3.createBucket({ Bucket: 'example' })
await s3.putObject({ Bucket: 'example', Body: 'Hello World!' })
Actually, looking at the code you're always using path style so it should be good!
Actually, looking at the code you're always using path style so it should be good!
You're looking at my generated code which doesn't get many API quirks baked in :) because the API definitions don't include them. My logic to upgrade S3 requests into virtual host style is here: https://github.com/cloudydeno/deno-aws_api/blob/6b267712cf00542603701a1fab333ede83707c6f/lib/client/client.ts#L67-L77
I already updated that heuristic to not enable when fixedEndpoint
is passed, and tested basic object ops with Minio. Unfortunately the s3 example in this repo doesn't work with Minio as-is. It seems like there's some incompatibility around bucket configuration APIs, which makes sense since targeting minio wasn't a goal of this library. If issues come up with usage I'll accept them as separate tickets.
The other two cases that I mention in this issue's text will be handled via a different mechanism.
Apparently minio runs a public test instance so you can try this out without any local setup other than Deno.
const minio = new ApiFactory({
fixedEndpoint: "https://play.min.io",
credentials: {
region: 'unused',
awsAccessKeyId: 'Q3AM3UQ867SPQQA43P2F',
awsSecretKey: 'zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG',
},
}).makeNew(S3);
Different types: