zitadel / zitadel-go

ZITADEL Go - The official client library of ZITADEL for an easy integration into your Go project.
https://zitadel.com
Apache License 2.0
65 stars 25 forks source link

[Bug] Local network communications #276

Open roquie opened 7 months ago

roquie commented 7 months ago

I want to do a simple thing. Close my development environment behind Cloudflare so no one can access, including the Zitadel test instance. But here's the trouble, by closing IP access to WAF I get 403 errors in the logs of my service which is integrated with Zitadel via this SDK.

In order to be able to use Zitadel within the local network (Kubernetes), I wrote a local GRPC endpoint address for the Zitadel service and at the Istio level, in sidecar, made a host replacement (to pass the security check).

This allowed the zitadel:80/.well-known/openid-configuration endpoint to converge successfully, similarly configured h2c app protocol in Kubernetes Service. Everything is done, for successful local operation.

My settings for the go-client: issuer: https://sso.example.com grpc_endpoint: zitadel:80 option: WithInsecure()

Log when trying to get a user profile:

  {"level":"error","timestamp":"2023-12-15T15:10:54Z","component":"usecase.getUserProfile","msg":"STOP. getZitadelUserProfile error","userID":"123131313312313123","error":"client.GetHumanProfile error: http status not ok: 403 Forbidden <!DOCTYPE html>\n<!--[if lt IE 7]> <html class=\"no-js ie6 oldie\" lang=\"en-US\"> <![endif]-->\n<!--[if IE 7]>    <html class=\"no-js ie7 oldie\" lang=\"en-US\"> <![endif]-->\n<!--[if IE 8]>    <html class=\"no-js ie8 oldie\" lang=\"en-US\"> <![endif]-->\n<!--[if gt IE 8]><!--> <html class=\"no-js\" lang=\"en-US\"> <!--<![endif]-->\n<head>\n<title>Attention Required! | Cloudflare</title>\n<meta charset=\"UTF-8\" />\n<meta http-equiv=\"Content-Type\" content=\"text/html; charset=UTF-8\" />\n<meta http-equiv=\"X-UA-Compatible\" content=\"IE=Edge\" />\n<meta name=\"robots\" content=\"noindex, nofollow\" />\n<meta name=\"viewport\" content=\"width=device-width,initial-scale=1\" />\n<link rel=\"stylesheet\" id=\"cf_styles-css\" href=\"/cdn-cgi/styles/cf.errors.css\" />\n<!--[if lt IE 9]><link rel=\"stylesheet\" id='cf_styles-ie-css' href=\"/cdn-cgi/styles/cf.errors.ie.css\" /><![endif]-->\n<style>body{margin:0;padding:0}</style>\n\n\n<!--[if gte IE 10]><!-->\n<script>\n  if (!navigator.cookieEnabled) {\n    window.addEventListener('DOMContentLoaded', function () {\n      var cookieEl = document.getElementById('cookie-alert');\n      cookieEl.style.display = 'block';\n    })\n  }\n</script>\n<!--<![endif]-->\n\n\n</head>\n<body>\n  <div id=\"cf-wrapper\">\n    <div class=\"cf-alert cf-alert-error cf-cookie-error\" id=\"cookie-alert\" data-translate=\"enable_cookies\">Please enable cookies.</div>\n    <div id=\"cf-error-details\" class=\"cf-error-details-wrapper\">\n      <div class=\"cf-wrapper cf-header cf-error-overview\">\n        <h1 data-translate=\"block_headline\">Sorry, you have been blocked</h1>\n        <h2 class=\"cf-subheadline\"><span data-translate=\"unable_to_access\">You are unable to access</span> example.com</h2>\n      </div><!-- /.header -->\n\n      <div class=\"cf-section cf-highlight\">\n        <div class=\"cf-wrapper\">\n          <div class=\"cf-screenshot-container cf-screenshot-full\">\n            \n              <span class=\"cf-no-screenshot error\"></span>\n            \n          </div>\n        </div>\n      </div><!-- /.captcha-container -->\n\n      <div class=\"cf-section cf-wrapper\">\n        <div class=\"cf-columns two\">\n          <div class=\"cf-column\">\n            <h2 data-translate=\"blocked_why_headline\">Why have I been blocked?</h2>\n\n            <p data-translate=\"blocked_why_detail\">This website is using a security service to protect itself from online attacks. The action you just performed triggered the security solution. There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.</p>\n          </div>\n\n          <div class=\"cf-column\">\n            <h2 data-translate=\"blocked_resolve_headline\">What can I do to resolve this?</h2>\n\n            <p data-translate=\"blocked_resolve_detail\">You can email the site owner to let them know you were blocked. Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.</p>\n          </div>\n        </div>\n      </div><!-- /.section -->\n\n      <div class=\"cf-error-footer cf-wrapper w-240 lg:w-full py-10 sm:py-4 sm:px-8 mx-auto text-center sm:text-left border-solid border-0 border-t border-gray-300\">\n  <p class=\"text-13\">\n    <span class=\"cf-footer-item sm:block sm:mb-1\">Cloudflare Ray ID: <strong class=\"font-semibold\">835f9db30d35250c</strong></span>\n    <span class=\"cf-footer-separator sm:hidden\">&bull;</span>\n    <span id=\"cf-footer-item-ip\" class=\"cf-footer-item hidden sm:block sm:mb-1\">\n      Your IP:\n      <button type=\"button\" id=\"cf-footer-ip-reveal\" class=\"cf-footer-ip-reveal-btn\">Click to reveal</button>\n      <span class=\"hidden\" id=\"cf-footer-ip\">209.38.252.7</span>\n      <span class=\"cf-footer-separator sm:hidden\">&bull;</span>\n    </span>\n    <span class=\"cf-footer-item sm:block sm:mb-1\"><span>Performance &amp; security by</span> <a rel=\"noopener noreferrer\" href=\"https://www.cloudflare.com/5xx-error-landing\" id=\"brand_link\" target=\"_blank\">Cloudflare</a></span>\n    \n  </p>\n  <script>(function(){function d(){var b=a.getElementById(\"cf-footer-item-ip\"),c=a.getElementById(\"cf-footer-ip-reveal\");b&&\"classList\"in b&&(b.classList.remove(\"hidden\"),c.addEventListener(\"click\",function(){c.classList.add(\"hidden\");a.getElementById(\"cf-footer-ip\").classList.remove(\"hidden\")}))}var a=document;document.addEventListener&&a.addEventListener(\"DOMContentLoaded\",d)})();</script>\n</div><!-- /.error-footer -->\n\n\n    </div><!-- /#cf-error-details -->\n  </div><!-- /#cf-wrapper -->\n\n  <script>\n  window._cf_translation = {};\n  \n  \n</script>\n\n</body>\n</html>\n","stacktrace":"gitlab.example2.com/common/go-libs/pkg/log.(*zapLogger).Error\n\t/build/vendor/gitlab.example2.com/common/go-libs/pkg/log/zap_logger.go:202\ngitlab.example2.com/pp/backend/internal/application/usecases/users/getUserProfile.(*UseCase).Run\n\t/build/internal/application/usecases/users/getUserProfile/usecase.go:76\ngitlab.example2.com/pp/backend/internal/application/gateways/openapi.(*Server).GetProfile\n\t/build/internal/application/gateways/openapi/getProfile.go:39\ngitlab.example2.com/pp/backend/internal/pkg/server/openapi/application.(*ServerInterfaceWrapper).GetProfile\n\t/build/internal/pkg/server/openapi/application/server.gen.go:666\ngithub.com/labstack/echo/v4.(*Echo).add.func1\n\t/build/vendor/github.com/labstack/echo/v4/echo.go:582\ngitlab.example2.com/common/go-libs/pkg/http/middlewares/recovery.EchoRecovery.func1.1\n\t/build/vendor/gitlab.example2.com/common/go-libs/pkg/http/middlewares/recovery/recovery.go:84\ngitlab.example2.com/common/go-libs/pkg/http/middlewares.Logging.EchoLogger.func2.1\n\t/build/vendor/gitlab.example2.com/common/go-libs/pkg/http/middlewares/logging.go:73\ngitlab.example2.com/common/go-libs/pkg/http/middlewares/metrics.Handler.EchoMetrics.func2.1.1\n\t/build/vendor/gitlab.example2.com/common/go-libs/pkg/http/middlewares/metrics/metrics.go:54\ngithub.com/slok/go-http-metrics/middleware.Middleware.Measure\n\t/build/vendor/github.com/slok/go-http-metrics/middleware/middleware.go:117\ngitlab.example2.com/common/go-libs/pkg/http/middlewares/metrics.Handler.EchoMetrics.func2.1\n\t/build/vendor/gitlab.example2.com/common/go-libs/pkg/http/middlewares/metrics/metrics.go:53\ngitlab.example2.com/common/go-libs/pkg/http/middlewares.ApiGatewayAuthMiddleware.func1.1\n\t/build/vendor/gitlab.example2.com/common/go-libs/pkg/http/middlewares/auth.go:137\ngithub.com/labstack/echo/v4.(*Echo).ServeHTTP\n\t/build/vendor/github.com/labstack/echo/v4/echo.go:669\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2938\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:2009"}

Now the question is why it ignores the local grpc endpoint and uses issuer to perform the request?

fforootd commented 7 months ago

I am not fully sure I can follow 😁 so let me try and ask some questions.

roquie commented 7 months ago

External access to zitadel works, right? (but you are fencing with CF)

It works, but we want to close it to the test environment behind an IP whitelist using WAF.

Accessing zitadel directly through the internal k8s server does return an error (should be instance not found)

It used to return an error, but now it doesn't, because I'm substituting the Host header at the sidecar (istio) level. By doing so, I just bypassed the zitadel check to try to use the service entirely inside the Kubernetes network.

The problem is that by closing the zitadel service behind CF, my service's integration with the zitadel service breaks. And this is despite the fact that zitadel's GPRC address is listed as local (service name in kubernetes)!

HaimKortovich commented 4 months ago

Hi @roquie, did you find a workaround? I'm having the same issue

roquie commented 4 months ago

No. @fforootd any updates?

HaimKortovich commented 4 months ago

ok so I managed to do it by creating my own (and probably not very good solution)

func Discover(key []byte, discoverUrl string) func(issuer string, scopes []string) (oauth2.TokenSource, error) {
    return func(issuer string, scopes []string) (oauth2.TokenSource, error) {
        var machineKeyData MachineKey
        if err := json.Unmarshal(key, &machineKeyData); err != nil {
            return nil, err
        }
        signer, err := client.NewSignerFromPrivateKeyByte([]byte(machineKeyData.Key), machineKeyData.KeyID)
        if err != nil {
            return nil, err
        }
        source := &jwtProfileTokenSource{
            clientID:   machineKeyData.UserID,
            audience:   []string{issuer},
            signer:     signer,
            scopes:     scopes,
            httpClient: http.DefaultClient,
        }
        config, err := client.Discover(discoverUrl, http.DefaultClient)
        if err != nil {
            return nil, err
        }
        source.tokenEndpoint = config.TokenEndpoint
        return source, nil
    }
}

func GetDiscoveryConfig(issuer string, httpClient *http.Client, wellKnownUrl ...string) (*oidc.DiscoveryConfiguration, error) {
    wellKnown := strings.TrimSuffix(issuer, "/") + oidc.DiscoveryEndpoint
    if len(wellKnownUrl) == 1 && wellKnownUrl[0] != "" {
        wellKnown = wellKnownUrl[0]
    }
    req, err := http.NewRequest("GET", wellKnown, nil)
    if err != nil {
        return nil, err
    }
    discoveryConfig := new(oidc.DiscoveryConfiguration)
    err = httphelper.HttpRequest(httpClient, req, &discoveryConfig)
    if err != nil {
        return nil, err
    }
    return discoveryConfig, nil
}

and passing it to the client

        zitadel.WithJWTProfileTokenSource(Discover([]byte(machineKeyData), GetAPIUrl(zitadelCluster))),
HaimKortovich commented 4 months ago

@fforootd