Minio Support

[mitchellmaler]

Hello, I am trying to get this to work using Minio as the S3 backend but when I provide all the connection information I am getting an error. I have other things using this minio cluster which is working fine so it makes me think that there is either an issue with the s3 library or my connection string.

2018/07/03 18:21:43 [Error] Failed to backup key data: RequestError: send request failed
caused by: Put http://consul-backups.minio.endpoint.com/: read tcp 10.1.0.147:39184->10.160.7.199:80: read: connection reset by peer

Here is my s3 file connection string.

-file="s3://key:token@consul-backups/consul_kvs.bak?region=default&endpoint=http://minio.endpoint.com&secure=false"

[aaronhurt]

The endpoint parameter is just a host and port. Try something like ...

-file="s3://key:token@consul-backups/consul_kvs.bak?region=default&endpoint=minio.endpoint.com&secure=false"

That will attempt an HTTP based connection to minio.endpoint.com on port 80. If you're minio instance uses a different port you will need to specify it in the endpoint parameter.

[mitchellmaler]

I just tried it without the http:// and with secure=fase and I am still getting the same issue. It seems to still be trying an https/443 connection

...&endpoint=minio.endpoint.com&secure=false"

I can see it is returning the error below which has

caused by: Put https://minio.endpoint.com/: dial tcp 10.160.7.199:443: connect: connection refused

I also tried specifying port 80 and it still tried the https protocol.

caused by: Put https://minio.endpoint.com:80/: read tcp 10.1.0.147:40886->10.160.7.199:80: read: connection reset by peer

This is all with the &secure=false

[aaronhurt]

So, I think I picked a really bad name for that parameter when reading the code:

https://github.com/myENA/consul-backinator/blob/v1.6.4/common/s3util.go#L105-L113

That will disable ssl/https if secure is set to true ... that's probably the exact opposite intent. I'll make a change to switch that logic.

[aaronhurt]

Please test the latest master branch and let me know if this corrects the behavior. Please re-open the issue if it does not.

[aaronhurt]

I can also roll a new release if needed.

[mitchellmaler]

Ah that makes more sense! yea it defiantly is a bit confusing. I looked at the code and just checking for false looks good but I am not sure if that is my underlying issue.

It is now using http and port 80 but I am still getting the error. Looking at it a bit closely it seems to be trying to hit the DNS of http://consul-backups.minio.endpoint.com/ on that IP address. Right now our minio server doesn't support DNS based buckets and only path based. I wonder if that is the main issue I am having.

[aaronhurt]

That could be it as well, I can add a path style option as well.

[aaronhurt]

Actually ... I forgot, I have that. It was fixed with PR #42 but it only lives in master:

https://github.com/myENA/consul-backinator/blob/master/common/s3util.go#L118-L126

[aaronhurt]

I'll roll a new release.

[mitchellmaler]

Ah awesome!! thanks

[aaronhurt]

Done, v1.6.5 contains this fix and the pathstyle addition.

[mitchellmaler]

@leprechau That fixed it! Thanks for the quick help!

[aaronhurt]

Np, thank you!