WSO2am 3.2.0 Docker - Devportal and Publisher not reachable behind reverse proxy

[kbegrow]

Description:

Hello, we have problems to get the wso2am image running in our staging environment as soon as we put it behind a proxy. Carbon is reachable without issues.

• The container is deployed in host mode and ports 8243 and 9443 are open on the VM

Following issues appear:

• in carbon ( Home > Identity > Service Providers)  are neither listed /publisher or /devportal
• if I try to open /devportal, I only see the blue circle of loading screen
• if I use developer tools in chrome I see following errors regarding /devportal:

  Failed to load resource: the server responded with a status of 400 ()

  Error while receiving settings :  Error
  at index.js:581
  at c (runtime.js:45)
  at Generator._invoke (runtime.js:271)
  at Generator.E.forEach.t.<computed> [as next] (runtime.js:97)
  at i (asyncToGenerator.js:5)
  at s (asyncToGenerator.js:27)
  (anonymous) @ DevPortal.jsx:100

• /publisher fails under https://gw.staging.{domain-name}.com/publisher/services/auth/login with http 500 error

NGINX proxy conf:

upstream sslapi.staging.{domain-name}.com {
    server {node-ip-address}:9443;
}

upstream sslgw.staging.{domain-name}.com {
    server {node-ip-address}:8243;
}

server {
    listen 80;
    server_name api.staging.{domain-name}.com;
    rewrite ^/(.*) https://api.staging.{domain-name}.com/$1 permanent;

}

server {
    listen 443 ssl;
    server_name api.staging.{domain-name}.com;
    proxy_set_header X-Forwarded-Port 443;
    ssl_certificate /etc/nginx/ssl/{cert_name};
    ssl_certificate_key /etc/nginx/ssl/{key_name};
    location / {
            proxy_set_header X-Forwarded-Host $host;
            proxy_set_header X-Forwarded-Server $host;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header Host $http_host;
            proxy_read_timeout 5m;
            proxy_send_timeout 5m;
            proxy_pass https://sslapi.staging.{domain-name}.com;
        }

        access_log /etc/nginx/log/am/https/access.log;
        error_log /etc/nginx/log/am/https/error.log;
}

server {
    listen 443 ssl;
    server_name gw.staging.{domain-name}.com;
    proxy_set_header X-Forwarded-Port 443;
    ssl_certificate /etc/nginx/ssl/{cert_name};
    ssl_certificate_key /etc/nginx/ssl/{key_name};
    location / {
            proxy_set_header X-Forwarded-Host $host;
            proxy_set_header X-Forwarded-Server $host;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header Host $http_host;
            proxy_read_timeout 5m;
            proxy_send_timeout 5m;
            proxy_pass https://sslgw.staging.{domain-name}.com;
        }

        access_log /etc/nginx/log/gw/https/access.log;
        error_log /etc/nginx/log/gw/https/error.log;
}

deployment.toml has following changes:

[server]
hostname = "api.staging.{domain-name}.com"
node_ip = "127.0.0.1"
#offset=0
mode = "single" #single or ha
#base_path = "${carbon.protocol}://${carbon.host}"
discard_empty_caches = false
server_role = "default"

[transport.https.properties]
proxyPort = 443

[[apim.gateway.environment]]
name = "Production and Sandbox"
type = "hybrid"
display_in_api_console = true
description = "This is a hybrid gateway that handles both production and sandbox token traffic."
show_as_token_endpoint_url = true
service_url = "https://localhost:${mgt.transport.https.port}/services/"
username= "${admin.username}"
password= "${admin.password}"
ws_endpoint = "ws://gw.staging.{domain-name}:9099"
wss_endpoint = "wss://gw.staging.{domain-name}:8099"
http_endpoint = "http://gw.staging.{domain-name}:${http.nio.port}"
https_endpoint = "https://gw.staging.{domain-name}:${https.nio.port}"

[apim.devportal]
url = "https://api.staging.{domain-name}.com/devportal"

docker logs shows following error during start up:

[2020-09-09 11:59:37,431] ERROR - [bridgeservlet] Servlet.service() for servlet [bridgeservlet] in context with path [/] threw exception
java.lang.NullPointerException: null
        at org.wso2.carbon.ui.tracker.AuthenticatorRegistry.getCarbonAuthenticator(AuthenticatorRegistry.java:67) ~[org.wso2.carbon.ui_4.6.0.jar:?]
        at org.wso2.carbon.ui.CarbonUILoginUtil.getAuthenticator(CarbonUILoginUtil.java:74) ~[org.wso2.carbon.ui_4.6.0.jar:?]
        at org.wso2.carbon.ui.CarbonSecuredHttpContext.handleSecurity(CarbonSecuredHttpContext.java:76) ~[org.wso2.carbon.ui_4.6.0.jar:?]
        at org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:60) ~[?:?]
        at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128) ~[?:?]
        at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:76) ~[?:?]
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:741) ~[tomcat-servlet-api_9.0.31.wso2v1.jar:?]
        at org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68) ~[org.wso2.carbon.tomcat.ext_4.6.0.jar:?]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) ~[tomcat_9.0.31.wso2v1.jar:?]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat_9.0.31.wso2v1.jar:?]
        at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) ~[tomcat_9.0.31.wso2v1.jar:?]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat_9.0.31.wso2v1.jar:?]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat_9.0.31.wso2v1.jar:?]
        at org.owasp.csrfguard.CsrfGuardFilter.doFilter(CsrfGuardFilter.java:72) ~[csrfguard_3.1.0.wso2v3.jar:?]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat_9.0.31.wso2v1.jar:?]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat_9.0.31.wso2v1.jar:?]
        at org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:65) ~[org.wso2.carbon.tomcat.ext_4.6.0.jar:?]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat_9.0.31.wso2v1.jar:?]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat_9.0.31.wso2v1.jar:?]
        at org.apache.catalina.filters.HttpHeaderSecurityFilter.doFilter(HttpHeaderSecurityFilter.java:126) ~[tomcat_9.0.31.wso2v1.jar:?]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat_9.0.31.wso2v1.jar:?]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat_9.0.31.wso2v1.jar:?]
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202) [tomcat_9.0.31.wso2v1.jar:?]
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) [tomcat_9.0.31.wso2v1.jar:?]
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541) [tomcat_9.0.31.wso2v1.jar:?]
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139) [tomcat_9.0.31.wso2v1.jar:?]
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) [tomcat_9.0.31.wso2v1.jar:?]
        at org.wso2.carbon.identity.context.rewrite.valve.TenantContextRewriteValve.invoke(TenantContextRewriteValve.java:86) [org.wso2.carbon.identity.context.rewrite.valve_1.4.0.jar:?]
        at org.wso2.carbon.identity.authz.valve.AuthorizationValve.invoke(AuthorizationValve.java:110) [org.wso2.carbon.identity.authz.valve_1.4.0.jar:?]
        at org.wso2.carbon.identity.auth.valve.AuthenticationValve.invoke(AuthenticationValve.java:75) [org.wso2.carbon.identity.auth.valve_1.4.0.jar:?]
        at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:99) [org.wso2.carbon.tomcat.ext_4.6.0.jar:?]
        at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:49) [org.wso2.carbon.tomcat.ext_4.6.0.jar:?]
        at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:62) [org.wso2.carbon.tomcat.ext_4.6.0.jar:?]
        at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:145) [org.wso2.carbon.tomcat.ext_4.6.0.jar:?]
        at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:688) [tomcat_9.0.31.wso2v1.jar:?]
        at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:57) [org.wso2.carbon.tomcat.ext_4.6.0.jar:?]
        at org.wso2.carbon.tomcat.ext.valves.RequestCorrelationIdValve.invoke(RequestCorrelationIdValve.java:119) [org.wso2.carbon.tomcat.ext_4.6.0.jar:?]
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) [tomcat_9.0.31.wso2v1.jar:?]
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) [tomcat_9.0.31.wso2v1.jar:?]
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:367) [tomcat_9.0.31.wso2v1.jar:?]
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) [tomcat_9.0.31.wso2v1.jar:?]
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868) [tomcat_9.0.31.wso2v1.jar:?]
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1639) [tomcat_9.0.31.wso2v1.jar:?]
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat_9.0.31.wso2v1.jar:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat_9.0.31.wso2v1.jar:?]
        at java.lang.Thread.run(Thread.java:834) [?:?]

Steps to reproduce:

Affected Product Version:

Environment details (with versions):

• OS: Ubuntu 18.04
• Client: VM
• Env (Docker/K8s): Docker

---

Optional Fields

Related Issues:

Suggested Labels:

Suggested Assignees:

[frshub-support]

Yes we have the same issue, please help asap. Thx

[wzamorski]

Anyone has a fix for that issue? I am configuring wso2am behind reverse proxy, seems to be a bug in the latest version... :(

[christianrhoades]

Did anyone ever find a fix for this issue? I am using NGINX as my webserver