search

percona / everest

Percona Everest is a cloud-native database platform to deploy and manage enterprise-grade PostgreSQL, MongoDB and MySQL database clusters.
https://docs.percona.com/everest/index.html
Apache License 2.0
92 stars 7 forks source link

S3 Compatible Storage from Dell Unable to Connect #352

Open atharmahboob opened 1 month ago

atharmahboob commented 1 month ago

I am using Percona Everest 0.10.1. Installation using an RKE-1 kubernetes cluster set up over a bunch of OpenStack VMs established using Rancher went successfully.

Percona Everest services are running successfully as I am able to get into Percona Everest web interface successfully.

When I try to create S3 compatible storage it gives error "Unable to connect. Check your credentials".

I am using settings with: (1) Using Path Style URL access; and (2) Disable Verify TLS Certificate

However, with the same set of credentials and same connection settings I am able to connect and successfully work with the same storage using S3 client tools like s3cmd and s3-explorer.

The S3 storage backend is Dell Storage.

recharte commented 1 month ago

Hi @atharmahboob thank you for reporting this. Can you share the logs of the Percona Everest API server pod? kubectl logs pods/percona-everest-xxxxxxxxxx-xxxxx -n everest-system

atharmahboob commented 1 month ago

This is the log entry I get when trying to connect:

{"level":"error","T":"2024-06-04T21:36:09Z","caller":"api/validation.go:209","msg":"RequestError: send request failed\ncaused by: Head \"https://s3.rapidcompute.com:80/test-bucket\": http: server gave HTTP response to HTTPS client","stacktrace":"github.com/percona/everest/api.s3Access\n\t/home/runner/work/everest/everest/api/validation.go:209\ngithub.com/percona/everest/api.validateStorageAccessByCreate\n\t/home/runner/work/everest/everest/api/validation.go:161\ngithub.com/percona/everest/api.validateCreateBackupStorageRequest\n\t/home/runner/work/everest/everest/api/validation.go:420\ngithub.com/percona/everest/api.(EverestServer).CreateBackupStorage\n\t/home/runner/work/everest/everest/api/backup_storage.go:71\ngithub.com/percona/everest/api.(ServerInterfaceWrapper).CreateBackupStorage\n\t/home/runner/work/everest/everest/api/everest-server.gen.go:1478\ngithub.com/oapi-codegen/echo-middleware.OapiRequestValidatorWithOptions.func1.1\n\t/home/runner/go/pkg/mod/github.com/oapi-codegen/echo-middleware@v1.0.1/oapi_validate.go:105\ngithub.com/percona/everest/api.(EverestServer).authenticate-fm.(EverestServer).authenticate.func1\n\t/home/runner/work/everest/everest/api/auth.go:51\ngithub.com/labstack/echo/v4.(Echo).add.func1\n\t/home/runner/go/pkg/mod/github.com/labstack/echo/v4@v4.11.4/echo.go:582\ngithub.com/labstack/echo/v4/middleware.LoggerWithConfig.func2.1\n\t/home/runner/go/pkg/mod/github.com/labstack/echo/v4@v4.11.4/middleware/logger.go:126\ngithub.com/percona/everest/api.NewEverestServer.RateLimiter.RateLimiterWithConfig.func1.1\n\t/home/runner/go/pkg/mod/github.com/labstack/echo/v4@v4.11.4/middleware/rate_limiter.go:148\ngithub.com/labstack/echo/v4.(Echo).ServeHTTP.func1\n\t/home/runner/go/pkg/mod/github.com/labstack/echo/v4@v4.11.4/echo.go:663\ngithub.com/percona/everest/api.(EverestServer).initHTTPServer.RemoveTrailingSlash.RemoveTrailingSlashWithConfig.func5.1\n\t/home/runner/go/pkg/mod/github.com/labstack/echo/v4@v4.11.4/middleware/slash.go:118\ngithub.com/labstack/echo/v4.(Echo).ServeHTTP\n\t/home/runner/go/pkg/mod/github.com/labstack/echo/v4@v4.11.4/echo.go:669\nnet/http.serverHandler.ServeHTTP\n\t/opt/hostedtoolcache/go/1.22.3/x64/src/net/http/server.go:3137\nnet/http.(*conn).serve\n\t/opt/hostedtoolcache/go/1.22.3/x64/src/net/http/server.go:2039"} ubuntu@k8s-dn:~$

atharmahboob commented 1 month ago

And this :

{"level":"error","T":"2024-06-04T21:42:25Z","caller":"api/validation.go:209","msg":"RequestError: send request failed\ncaused by: Head \"https://s3.rapidcompute.com:80/test-bucket\": http: server gave HTTP response to HTTPS client","stacktrace":"github.com/percona/everest/api.s3Access\n\t/home/runner/work/everest/everest/api/validation.go:209\ngithub.com/percona/everest/api.validateStorageAccessByCreate\n\t/home/runner/work/everest/everest/api/validation.go:161\ngithub.com/percona/everest/api.validateCreateBackupStorageRequest\n\t/home/runner/work/everest/everest/api/validation.go:420\ngithub.com/percona/everest/api.(EverestServer).CreateBackupStorage\n\t/home/runner/work/everest/everest/api/backup_storage.go:71\ngithub.com/percona/everest/api.(ServerInterfaceWrapper).CreateBackupStorage\n\t/home/runner/work/everest/everest/api/everest-server.gen.go:1478\ngithub.com/oapi-codegen/echo-middleware.OapiRequestValidatorWithOptions.func1.1\n\t/home/runner/go/pkg/mod/github.com/oapi-codegen/echo-middleware@v1.0.1/oapi_validate.go:105\ngithub.com/percona/everest/api.(EverestServer).authenticate-fm.(EverestServer).authenticate.func1\n\t/home/runner/work/everest/everest/api/auth.go:51\ngithub.com/labstack/echo/v4.(Echo).add.func1\n\t/home/runner/go/pkg/mod/github.com/labstack/echo/v4@v4.11.4/echo.go:582\ngithub.com/labstack/echo/v4/middleware.LoggerWithConfig.func2.1\n\t/home/runner/go/pkg/mod/github.com/labstack/echo/v4@v4.11.4/middleware/logger.go:126\ngithub.com/percona/everest/api.NewEverestServer.RateLimiter.RateLimiterWithConfig.func1.1\n\t/home/runner/go/pkg/mod/github.com/labstack/echo/v4@v4.11.4/middleware/rate_limiter.go:148\ngithub.com/labstack/echo/v4.(Echo).ServeHTTP.func1\n\t/home/runner/go/pkg/mod/github.com/labstack/echo/v4@v4.11.4/echo.go:663\ngithub.com/percona/everest/api.(EverestServer).initHTTPServer.RemoveTrailingSlash.RemoveTrailingSlashWithConfig.func5.1\n\t/home/runner/go/pkg/mod/github.com/labstack/echo/v4@v4.11.4/middleware/slash.go:118\ngithub.com/labstack/echo/v4.(Echo).ServeHTTP\n\t/home/runner/go/pkg/mod/github.com/labstack/echo/v4@v4.11.4/echo.go:669\nnet/http.serverHandler.ServeHTTP\n\t/opt/hostedtoolcache/go/1.22.3/x64/src/net/http/server.go:3137\nnet/http.(*conn).serve\n\t/opt/hostedtoolcache/go/1.22.3/x64/src/net/http/server.go:2039"}

atharmahboob commented 1 month ago

Here is a screenshot of my connection settings: Screenshot from 2024-06-05 02-47-30

atharmahboob commented 1 month ago

I think Percon a Everest is constructing the connection URL with https:// instead of http// despite my selecting a non-TLS connection.

Please advise.

recharte commented 2 weeks ago

@atharmahboob if you want to connect to your storage server using http instead of https you should explicitly set it in the endpoint URL like so http://s3.rapidcompute.com.

Disabling the verify TLS certificate option doesn't assume http. Instead, it connects using https but skips the certificate validation. The intended use case for unsetting this option is when the storage server is using a self-signed certificate but you still you want to connect using https.