Cingulara / openrmf-docs

Documentation on the OpenRMF application, including scripts to run the whole stack as well as just infrastructure with documentation on using the tool.
https://www.openrmf.io/
GNU General Public License v3.0
126 stars 27 forks source link

[BUG] SSL Permissions Issue #208

Closed alphillips-lab closed 3 years ago

alphillips-lab commented 3 years ago

Describe the bug Hello. Working to set up https on my containers and came across an issue similar to bug #160 by @medined. I get the error "There is an Authentication problem. Please logout and log back in. And have the application administrator verify your API's authentication settings."

The guide found in the docs was followed and I'm a bit at a loss here. I initially went looking at the firewall as mentioned in #160 but it doesn't seem to have any glaring issues either. No manual edits were made, but comparing the http version with the ssl version of iptables seems like it checks out alright to me.

To Reproduce So here's how I got to where I am:

At the moment, that's pretty much everything I've done. I'll post screenshot as well to get a better idea of what's going on. I'm doing this in a sandboxed text environment before I move it to prod so I'm not worried about sharing the default firewall rules either in case there's something I missed.

Note that I've tried both configuring the openrmf-web container to expose both 8080 and 8443 as well as only exposing 8443. Both have the same effect (as I would have expected).

Screenshots The error image

Firewall Rules (no other firewall enabled and no manual modifications made)
-P INPUT ACCEPT
-P FORWARD DROP
-P OUTPUT ACCEPT
-N DOCKER
-N DOCKER-ISOLATION-STAGE-1
-N DOCKER-ISOLATION-STAGE-2
-N DOCKER-USER
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-ISOLATION-STAGE-1
-A FORWARD -o br-3ffa6654a9b5 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o br-3ffa6654a9b5 -j DOCKER
-A FORWARD -i br-3ffa6654a9b5 ! -o br-3ffa6654a9b5 -j ACCEPT
-A FORWARD -i br-3ffa6654a9b5 -o br-3ffa6654a9b5 -j ACCEPT
-A FORWARD -o br-1953bf496d0b -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o br-1953bf496d0b -j DOCKER
-A FORWARD -i br-1953bf496d0b ! -o br-1953bf496d0b -j ACCEPT
-A FORWARD -i br-1953bf496d0b -o br-1953bf496d0b -j ACCEPT
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A DOCKER -d 172.23.0.2/32 ! -i br-3ffa6654a9b5 -o br-3ffa6654a9b5 -p tcp -m tcp --dport 5432 -j ACCEPT
-A DOCKER -d 172.23.0.3/32 ! -i br-3ffa6654a9b5 -o br-3ffa6654a9b5 -p tcp -m tcp --dport 8443 -j ACCEPT
-A DOCKER -d 172.22.0.2/32 ! -i br-1953bf496d0b -o br-1953bf496d0b -p tcp -m tcp --dport 3000 -j ACCEPT
-A DOCKER -d 172.22.0.3/32 ! -i br-1953bf496d0b -o br-1953bf496d0b -p tcp -m tcp --dport 16687 -j ACCEPT
-A DOCKER -d 172.22.0.3/32 ! -i br-1953bf496d0b -o br-1953bf496d0b -p tcp -m tcp --dport 16686 -j ACCEPT
-A DOCKER -d 172.22.0.4/32 ! -i br-1953bf496d0b -o br-1953bf496d0b -p tcp -m tcp --dport 8080 -j ACCEPT
-A DOCKER -d 172.22.0.5/32 ! -i br-1953bf496d0b -o br-1953bf496d0b -p tcp -m tcp --dport 8092 -j ACCEPT
-A DOCKER -d 172.22.0.6/32 ! -i br-1953bf496d0b -o br-1953bf496d0b -p tcp -m tcp --dport 7777 -j ACCEPT
-A DOCKER -d 172.22.0.3/32 ! -i br-1953bf496d0b -o br-1953bf496d0b -p tcp -m tcp --dport 14271 -j ACCEPT
-A DOCKER -d 172.22.0.7/32 ! -i br-1953bf496d0b -o br-1953bf496d0b -p tcp -m tcp --dport 9090 -j ACCEPT
-A DOCKER -d 172.22.0.8/32 ! -i br-1953bf496d0b -o br-1953bf496d0b -p tcp -m tcp --dport 8094 -j ACCEPT
-A DOCKER -d 172.22.0.9/32 ! -i br-1953bf496d0b -o br-1953bf496d0b -p tcp -m tcp --dport 7778 -j ACCEPT
-A DOCKER -d 172.22.0.10/32 ! -i br-1953bf496d0b -o br-1953bf496d0b -p tcp -m tcp --dport 27017 -j ACCEPT
-A DOCKER -d 172.22.0.11/32 ! -i br-1953bf496d0b -o br-1953bf496d0b -p tcp -m tcp --dport 27017 -j ACCEPT
-A DOCKER -d 172.22.0.12/32 ! -i br-1953bf496d0b -o br-1953bf496d0b -p tcp -m tcp --dport 8222 -j ACCEPT
-A DOCKER -d 172.22.0.13/32 ! -i br-1953bf496d0b -o br-1953bf496d0b -p tcp -m tcp --dport 27017 -j ACCEPT
-A DOCKER -d 172.22.0.14/32 ! -i br-1953bf496d0b -o br-1953bf496d0b -p tcp -m tcp --dport 27017 -j ACCEPT
-A DOCKER -d 172.22.0.3/32 ! -i br-1953bf496d0b -o br-1953bf496d0b -p udp -m udp --dport 6831 -j ACCEPT
-A DOCKER -d 172.22.0.15/32 ! -i br-1953bf496d0b -o br-1953bf496d0b -p tcp -m tcp --dport 27017 -j ACCEPT
-A DOCKER -d 172.22.0.12/32 ! -i br-1953bf496d0b -o br-1953bf496d0b -p tcp -m tcp --dport 6222 -j ACCEPT
-A DOCKER -d 172.22.0.3/32 ! -i br-1953bf496d0b -o br-1953bf496d0b -p tcp -m tcp --dport 5778 -j ACCEPT
-A DOCKER -d 172.22.0.12/32 ! -i br-1953bf496d0b -o br-1953bf496d0b -p tcp -m tcp --dport 4222 -j ACCEPT
-A DOCKER -d 172.22.0.16/32 ! -i br-1953bf496d0b -o br-1953bf496d0b -p tcp -m tcp --dport 8090 -j ACCEPT
-A DOCKER -d 172.22.0.17/32 ! -i br-1953bf496d0b -o br-1953bf496d0b -p tcp -m tcp --dport 8096 -j ACCEPT
-A DOCKER -d 172.22.0.18/32 ! -i br-1953bf496d0b -o br-1953bf496d0b -p tcp -m tcp --dport 8084 -j ACCEPT
-A DOCKER -d 172.22.0.19/32 ! -i br-1953bf496d0b -o br-1953bf496d0b -p tcp -m tcp --dport 8098 -j ACCEPT
-A DOCKER -d 172.22.0.20/32 ! -i br-1953bf496d0b -o br-1953bf496d0b -p tcp -m tcp --dport 8088 -j ACCEPT
-A DOCKER -d 172.22.0.29/32 ! -i br-1953bf496d0b -o br-1953bf496d0b -p tcp -m tcp --dport 8086 -j ACCEPT
-A DOCKER -d 172.22.0.30/32 ! -i br-1953bf496d0b -o br-1953bf496d0b -p tcp -m tcp --dport 8082 -j ACCEPT
-A DOCKER -d 172.22.0.31/32 ! -i br-1953bf496d0b -o br-1953bf496d0b -p tcp -m tcp --dport 8443 -j ACCEPT
-A DOCKER-ISOLATION-STAGE-1 -i br-3ffa6654a9b5 ! -o br-3ffa6654a9b5 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -i br-1953bf496d0b ! -o br-1953bf496d0b -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -j RETURN
-A DOCKER-ISOLATION-STAGE-2 -o br-3ffa6654a9b5 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -o br-1953bf496d0b -j DROP
-A DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -j RETURN
-A DOCKER-USER -j RETURN

Redirect in Client>Openrmf image

Docker Compose for openrmf image

.env File image

Keycloak docker compose image

Nginx Config image

Desktop (please complete the following information): This is an ubuntu 20.04 vm

degthat8412 commented 3 years ago

@alphillips-lab Did you change your client information within KeyCloak? image

degthat8412 commented 3 years ago

and in your .env file

alphillips-lab commented 3 years ago

and in your .env file

Yes, screenshot 2 (under firewall rules) and 4 would be what you are referring to.

degthat8412 commented 3 years ago

You may what to post this in our slack channel. We do have groups in there that have done what your asking for. They may have a quick answer.

On Wed, Jan 6, 2021 at 7:43 PM alphillips-lab notifications@github.com wrote:

and in your .env file

Yes, screenshot 2 (under firewall rules) and 4 would be what you are referring to.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Cingulara/openrmf-docs/issues/208#issuecomment-755802055, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJ7XVZY2Q4JQIRSIFTRYNCDSYT7RPANCNFSM4VYGVGKA .

-- David Gould CTO/ Cyber Security Professional DGOULD@tutelasec.com CISSP/CEH/MCSE/VCP

alphillips-lab commented 3 years ago

Just cross-referenced it there as well. Thanks for the heads up. I'll definitely post here if I come to any conclusions in the mean time but still a bit puzzled on this one at the moment...

Cingulara commented 3 years ago

Realize that the open source setup was to have this all running on one box by design. And over http locally. So there may be a few more places to check for https as well as for redirecting to/from Keycloak securely. I did that URL linked below 6 or 7 months ago so I am going off memory here. I will not have time to test this for a few days unfortunately.

I did not see any reference to this https://cingulara.github.io/openrmf-docs/domainkeycloak.html here. I believe to make an https://openrmf:xxxx/ redirect to keycloak correctly the auth.js would have to be overwritten to use https redirection as well. Just making sure that was done to check it off the list of things. You have to go to the https of keycloak to keep this all https.

You may also want to set this to require here for SSL: image

And want to make sure your https://xxx.xxx.xxx.xxx:9001/ JWT-AUTHORITY and https://xxx.xxx.xxx.xxx:8443/ you are using is an IP or that DNS name you put in and are the same. And that the xxx.xxx.xxx.xxx or DNS Name resolves from within the docker stack. You can do a docker exec -it name-of-container /bin/sh into one of the API or MSG clients to make sure a wget or curl or ping or something can get out. The web client gets the JWT from Keycloak, posts an API call to the backend APIs, passes the JWT, and then the JWT is validated.

Usually when I get that error at the very top here, it is because my IP I am going to does not match the keycloak JWT that authenticated me. The JS on each page that does a "is keycloak authenticated" is what that popup is referring to. That is also why your read, template, and other API calls get a 401. The JWT cannot be validated by the APIs.

The other thing to do would be to refresh that page, then copy out (and sanitize) your logs using docker logs name-of-container where the name-of-container is from a docker ps command. It will be something like scripts_openrmfapi-read_1 or something. The logs should put out any error messages we may be able to decipher.

alphillips-lab commented 3 years ago

Thanks for the reply! I had not overwritten the auth.js file, but after doing so it didn't change anything. The first screenshot looks to me like it's making requests to the right spots but just can't reach them with the 401. I am just using an ip address at the moment and everywhere I've touched is all using the same address at this point. I tried a curl from a handful of containers and all seemed to connect to the https site, but I'll try the whole list of api containers.

And note that every container that I've tried can access the site but all receive unauthorized responses when attempting to hit the endpoints. image

This is the output of the closest docker container I could find (openrmf-read-api) "openrmf" was replaced with the IP/DNS but just fyi, logs don't have any https or reference to port number here. Not sure if I even got the right logs for you.

rl: http://openrmf/count/artifacts|action: CountArtifacts
2021-01-07 13:37:46.1138|2|INFO|Microsoft.AspNetCore.Hosting.Diagnostics|Request finished in 573.5633ms 401 text/plain |url: http://openrmf/count/artifacts|action: CountArtifacts
2021-01-07 13:37:46.1175|13|ERROR|Microsoft.AspNetCore.Server.Kestrel|Connection id "0HM5J1T4SN1MN", Request id "0HM5J1T4SN1MN:00000001": An unhandled exception was thrown by the application. System.InvalidOperationException: StatusCode cannot be set because the response has already started.
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ThrowResponseAlreadyStartedException(String value)
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.set_StatusCode(Int32 value)
   at Microsoft.AspNetCore.Authentication.JwtBearer.JwtBearerHandler.HandleChallengeAsync(AuthenticationProperties properties)
   at Microsoft.AspNetCore.Authentication.AuthenticationHandler`1.ChallengeAsync(AuthenticationProperties properties)
   at Microsoft.AspNetCore.Authentication.AuthenticationService.ChallengeAsync(HttpContext context, String scheme, AuthenticationProperties properties)
   at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
   at Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware.Invoke(HttpContext httpContext)
   at Swashbuckle.AspNetCore.Swagger.SwaggerMiddleware.Invoke(HttpContext httpContext, ISwaggerProvider swaggerProvider)
   at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)
   at Prometheus.HttpMetrics.HttpRequestDurationMiddleware.Invoke(HttpContext context)
   at Prometheus.HttpMetrics.HttpRequestCountMiddleware.Invoke(HttpContext context)
   at Prometheus.HttpMetrics.HttpInProgressMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Builder.Extensions.MapMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)|url: http://openrmf/count/systems|action: CountSystems
2021-01-07 13:37:46.1175|13|ERROR|Microsoft.AspNetCore.Server.Kestrel|Connection id "0HM5J1T4SN1MN", Request id "0HM5J1T4SN1MN:00000001": An unhandled exception was thrown by the application. System.InvalidOperationException: StatusCode cannot be set because the response has already started.
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ThrowResponseAlreadyStartedException(String value)
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.set_StatusCode(Int32 value)
   at Microsoft.AspNetCore.Authentication.JwtBearer.JwtBearerHandler.HandleChallengeAsync(AuthenticationProperties properties)
   at Microsoft.AspNetCore.Authentication.AuthenticationHandler`1.ChallengeAsync(AuthenticationProperties properties)
   at Microsoft.AspNetCore.Authentication.AuthenticationService.ChallengeAsync(HttpContext context, String scheme, AuthenticationProperties properties)
   at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
   at Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware.Invoke(HttpContext httpContext)
   at Swashbuckle.AspNetCore.Swagger.SwaggerMiddleware.Invoke(HttpContext httpContext, ISwaggerProvider swaggerProvider)
   at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)
   at Prometheus.HttpMetrics.HttpRequestDurationMiddleware.Invoke(HttpContext context)
   at Prometheus.HttpMetrics.HttpRequestCountMiddleware.Invoke(HttpContext context)
   at Prometheus.HttpMetrics.HttpInProgressMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Builder.Extensions.MapMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)|url: http://openrmf/count/systems|action: CountSystems
2021-01-07 13:37:46.1269|2|INFO|Microsoft.AspNetCore.Hosting.Diagnostics|Request finished in 565.0684ms 401 text/plain |url: http://openrmf/count/systems|action: CountSystems
alphillips-lab commented 3 years ago

I looked into the firewall a bit more and I can see that one rule is still persisting for a port 8080. Did a miss a web server on a container? Do I need to have both 8443 AND 8080 open on the openrmf-web container?

I'm not sure how relevant this is because it looks like this is an entirely different container based on the different IP but I don't know enough about the way OpenRMF is built. Figured I'd post it here for visibility nevertheless.

Attached are a couple of screenshots dealing with this

Contextual Diff image

List of ports in use on firewall (sorted and filtered everything out obviously, 3 8080s is http and 1 8080 and 2 8443 is https attempt) image

HTTPS firewall ports of interest image

HTTP ports of interest image

alphillips-lab commented 3 years ago

just following up on this one, as I still haven't come to any meaningful conclusion. I can see that it most likely has to do with some reason where the nginx server receives the https traffic and can't proxy to the http interface on the api containers properly. My understanding was that the "proxy_pass" with the http url should be sufficient to accomplish this. Does each container need to be running the ssl protocol?

My current setup and expectation was something like this https -----> nginx container ----> http ---->api containers

but it looks like this seems to create the errors as seen in those logs beforehand and break the chain somewhere between the nginx container and the upstream containers.

appreciate the input, thanks

Cingulara commented 3 years ago

I just got back in town so I will be trying to duplicate how I did this several months ago to find an answer out.

Cingulara commented 3 years ago

I believe it may be with Keycloak having to run in SSL mode, doing a "require SSL" on the realm login settings for OpenRMF, making NGINX run on 8443, and the .env having the backend APIs validate the JWT from the Keycloak server they are all setup to use. The web UI redirects to Keycloak, logs you in, and then gets a JWT token in essence to carry around and call APIs. The APIs validate the JWT on every single call. The name of the server the web UI redirects to has to match the .env path to Keycloak as well. If using https, has to do https. If using 192.168.11.13 for the web UI, the .env has to match the exact 192.168.11.13 so they validate the JWT.

I think that part of validating the JWT is the problem. I can setup my Keycloak to use a local 192.168.x.x address in my Keycloak setup and my .env, but then if I go to my local OpenRMF with http://localhost:8080/ it will fail even if I allow localhost:8080/* in my Valid URIs setup in Keycloak. The way you go to OpenRMF and Keycloak via IP or Name has to match. And HTTPS adds that into the mix. That is my thinking.

I am setting this up on my box now and in the middle of doing other things. I have to destroy my setup and save it off, then start over for this so I don't mess up my local dev setup. Working on that now as the SSL thing is invasive to setup for Keycloak and OpenRMF and the NGINX conf and the api.js and such.

Cingulara commented 3 years ago

I updated my local stack in OpenRMF to use 8443 and the generated self signed certs I did following the steps in https://nickolaskraus.org/articles/how-to-create-a-self-signed-certificate-for-nginx-on-macos/ since I have a Mac. I got all that to work and am updating the online docs for HTTPS to include more info.

The OpenRMF Keycloak.js automatically redirects to https://xxx.xxx.xxx.xxx:9001/auth/reamls/openrmf/protocol/openid-connect/auth?xxxxxx with a bunch of stuff using the port 9001 and HTTPS. It does this based on the Javascript in the ./js/auth.js file I updated a few months ago. You can keep 9001 and through docker-compose in the backend map 9001 to HTTPS Keycloak on 8443, or you can change the port in the docker-compose and mount a js/auth.js with a new port as that port is hard coded for now. The protocol and path and port are automatically assumed in the JS of the OpenRMF application.

I am working now to setup Keycloak on HTTPS and then make the docker-compose of Keycloak run over 8443 and HTTPS. Then let Keycloak use the new valid URI for OpenRMF and test it out to make sure it works.

Cingulara commented 3 years ago

@alphillips-lab Alternatively IF you wanted to, you could actually using the same NGINX setup to front Keycloak on port 8443 with a path like "/keycloak" once you have HTTPS working for you in NGINX. If NGINX is working right, anything it proxies should be OK as well really. You could hide Keycloak behind NGINX, and make OpenRMF redirect to it with it fronted with NGINX. And the Valid URI would need to allow it.

That is a little more invasive and changes the architecture a bit. But I believe it would work. I have done something similar on the updated OpenRMF Professional we are working on right now. You would have to combine some of the docker-compose files, update NGINX to use Keycloak behind it, etc. And change the auth.js by mounting a local file to replace it and have the port/path updated. Your ENV also would need to point to the same thing of course. This could allow one single docker-compose to run it all and then you only setup NGINX to use HTTPS to make it a bit simpler.

But that would let you run it all on 8443 and have just 1 port open. This may be a good idea to do for a future version. The challenge being if you change where Keycloak is running, the mounted volume disappears and you have to set it up again. So that is a PITA.

alphillips-lab commented 3 years ago

@Cingulara Thanks for your replies

Regarding your first statement, I've been aware about how the redirects function, using relative path as the baseline for the next redirect to the server. My .env file calls to the local IP address along with HTTPS prepended. In addition, I've been testing the connectivity using another VM so I always will have to access the machine using its address. My screenshots of the main error look to me like it's trying to access to endpoints with the HTTPS prefix and the correct port, along with the right IP address so it doesn't appear to be a mismatch there.

I also made sure that keycloak was running HTTPS, using your setup to external port 9001 and switch internal port through the docker-compose file to port 8443. I also set up "require ssl" in the realm, but tried the other options to see if it had any effect. No go.

I'm not sure how much it would help to put keycloak behind the same external port since it appears to me that the endpoints can reach port 9001 from inside to stack anyways, but if you were able to get it to work that way I'm up for anything.

Were you able to get your setup working yet? Was there anything specifically that you did differently from myself? At the moment it appears that we've done pretty much the same things based on your replies.

Cingulara commented 3 years ago

Yes I did get my setup work work 95% of the way. I setup NGINX w/ the same information for OpenRMF as I did for the Keycloak setup just to get HTTPS working for Keycloak in an easier manner. I can go to HTTPS on Keycloak and set it up. And then HTTPS on OpenRMF, have it redirect w/o any mods to ./js/auth.js, and then come back. The last API calls are 401 errors.

The last step is to make sure all APIs know the HTTPS is a valid certificate otherwise they will fail the behind-the-scenes request. I am using a self signed cert. So the last step of the API talking to the Keycloak backend server over HTTPS with a self-signed cert fails as it does not know the cert is valid. If I can get that working, we are good. I had the same issue w/ the S.C. gov't folks as it did not recognize DoD certs being valid. We had to allow that and they were good. I need to do the same thing locally.

System.Security.Authentication.AuthenticationException: The remote certificate is invalid according to the validation procedure.
Cingulara commented 3 years ago

I am going to post updated docs and how I did this with NGINX and a modified docker-compose. You have to have a valid cert that can be validated (not self-signed and not your own CA) to make it work the way I am going to update the docs. I do not want to modify code to allow self-signed certs. I need to see a way to make self-signed certs allowed dynamically in the API containers w/o redoing them. Adding the certificate in the browser is easy. I did that this morning. But the behind the scenes API validating the JWT token w/ the Keycloak server running HTTPS is the last sticking point.

alphillips-lab commented 3 years ago

I had a passing thought that it could be that the containers just weren't set up to validate self-signed certs. A little tough for myself since I was testing with a self-signed certs but prod would move to an internal CA, both of which you've described wouldn't work with the current architecture.

Was there a particular path used to get the DoD certs working from their CA? Maybe it's too complex to share but provided I at least had a general idea of where to look in the code I wouldn't mind forking and making an attempt because it's pretty important to me.

Otherwise, any other way to possibly get a secure transfer of data between two different boxes that you might recommend? I'm figuring I might try and set up a second proxy that just leaves openrmf untouched (HTTP mode) and any external requests just get routed through that, but I'm not really sure if that would even work.

I appreciate your help with everything. I am definitely loving this project as it's the first RMF I'd be doing with it and it seems like any other means of tracking is vastly inferior. Really awesome project!

Cingulara commented 3 years ago

OK got it working 100% even with self-signed certs. The self-signed certs to have be mounted with the CRT and Key file if you want the APIs to work. OR you just use a valid CA per normal and do it that way. I believe you also can generate a valid CA, make the CRTs and then mount the CA into the same folders. That gives you options. I took valid CRT files and put into the /usr/local/share/ca-certificates/ and ran install ca-certificates to make my base image to NOT do this. But for SSL certs later on, you can.

    volumes:
      - ./ssl/certs/self-signed.crt:/etc/ssl/certs/self-signed.crt.conf:ro
      - ./ssl/private/self-signed.key:/etc/ssl/private/self-signed.key.conf:ro

I am going to post my updated SSL docs shortly so it can be followed with more examples. Thanks for asking and pushing this!

Cingulara commented 3 years ago

Great Minds think alike @alphillips-lab you got it. They were not setup to do self-signed certs. This "fix" or better "workaround" above lets you. I am running it locally as I type this.

Cingulara commented 3 years ago

All running SSL locally, fronting by NGINX, Keycloak fronted by NGINX, using self-signed certs.

image

Cingulara commented 3 years ago

https://cingulara.github.io/openrmf-docs/https.html has more information.

alphillips-lab commented 3 years ago

Ok, this is great news! Bad news for me is that it doesn't seem to be working...

Made a fresh install so I don't accidentally have any misconfigurations. I then followed your steps, making a proxy for keycloak that uses the same SSL certs that I generated for this instance. Called everything in the same way as you using snippets, and added the ssl certs to each of the 9 api container volumes. Still no dice, getting the same 401 error after logging in. I'm going to post all of my screenshots relating to key points of interest, and let me know if I'm missing anything... Note that I used the same naming conventions as you so there should be no confusion. This one is a local IP so I'll just leave it in to give you a better idea of the full context.

OPENRMF

.env File

JWT-AUTHORITY=https://192.168.75.207:9001/auth/realms/openrmf
JWT-CLIENT=openrmf

Web Container image

Volumes added to api containers image

Openrmf NGINX Config for redirect and SSL image

Snippets

ssl_certificate /etc/nginx/certs/ssl/certs/self-signed.crt;
ssl_certificate_key /etc/nginx/certs/ssl/private/self-signed.key;
ssl_protocols TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH";
ssl_ecdh_curve secp384r1;
ssl_session_cache shared:SSL:10m;
ssl_session_tickets off;
ssl_stapling on;
ssl_stapling_verify on;
resolver 8.8.8.8 8.8.4.4 valid=300s;
resolver_timeout 5s;
add_header Strict-Transport-Security "max-age=63072000; includeSubdomains";
add_header X-Frame-Options DENY;
add_header X-Content-Type-Options nosniff;
ssl_dhparam /etc/nginx/certs/ssl/certs/dhparam.pem;

SSL directory that I push into the containers image

auth.js

var keycloak = Keycloak({
 url: 'https://192.168.75.207:9001/auth/',
 realm: 'openrmf',
 clientId: 'openrmf'
});

KEYCLOAK

snippets and certs are the same for both

Changes to the keycloak docker-compose file

keycloakproxy:
    image: nginxinc/nginx-unprivileged:1.18-alpine
    restart: always
    ports:
      - 9001:8443
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
      - ./snippets/:/etc/nginx/snippets/:ro
      - ./ssl/:/etc/nginx/certs/ssl/:ro
    networks:
      - keycloak-network
    depends_on:
      - keycloak

  keycloak:
    image: jboss/keycloak:10.0.2
    restart: always
    ports:
      - 8080
... ...
...

Keycloak NGINX Proxy config

worker_processes 4;

pid /tmp/nginx.pid; # Changed from /var/run/nginx.pid

events { worker_connections 4096; }

http {

    sendfile on;
    client_max_body_size 60M;
    include /etc/nginx/mime.types;
    keepalive_timeout  65;
    proxy_http_version 1.1;

    server {
        listen 8443 ssl;
        server_name 192.168.75.207;
        include snippets/self-signed.conf;
        include snippets/ssl-params.conf;

        # # don't send the nginx version number in error pages and Server header
        server_tokens off;

        proxy_set_header        Host $host:9001;
        proxy_set_header        X-Real-IP $remote_addr;
        proxy_set_header        X-Real-IP $remote_addr;
        # proxy_set_header        X-Forwarded-For $host;
        proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header        X-Forwarded-Proto $scheme;

        location / {
            proxy_pass http://keycloak:8080;
            add_header X-Frame-Options "ALLOWALL";
        }
    }
}

Require SSL on Realm image

Valid Redirect URIs image

I'm at a big loss considering I'm pretty certain I retraced your footsteps in every, but I'm still coming up short.

openssl req -x509 -nodes -days 365 -newkey rsa:4096 -keyout self-signed.key -out self-signed.crt
openssl dhparam -out dhparam.pem 2048

Was there anything else you changed that I'm missing?

Cingulara commented 3 years ago

I did not modify my auth.js. Based on making Keycloak https and keeping it at port 9001, the regular auth.js will read that and redirect accordingly. You should not need that.

What is the error showing up? Is it a 401 on the main dashboard if you view the Javascript Console? I am guessing 401 Unauthorized?

Can you log into Keycloak over the https://192.168.75.207:9001/auth/ correctly and get in, see the OpenRMF realm, view users, etc? I spent a bit of time after setup making sure that worked. If you can go to the OpenRMF realm and then make sure there are roles, users, etc. and that the redirect is right we should be OK there. It looks like you did from the screenshots above.

Clear cache on the web browser as well just to make sure.

Also I want to make sure your user account for this app in Keycloak has a role assigned to it, even if Reader, but better Administrator, so we can make sure the account is setup right.

This may be worth a google meet / webex screenshare so we don't go back and forth a bunch of times. Then we document the fix here for everyone else. I have my setup locally and did not check in any of my updates so I know what files were changed on purpose. Just so I can trace this if there is an issue.

Nothing jumps at me right now on this unfortunately. I just spun mine up again and it is still running correctly.

alphillips-lab commented 3 years ago

@Cingulara All working now after some additional troubleshooting... for other users if they happen to come across this it was a config issue/mismatch between what the key was expecting in the metadata vs what the key was pushing in the metadata (issuer, CN, location, etc.)

Ironically fixed by filling everything in and not leaving ANYTHING default (just did CN at first).

Can't thank you enough for the help! I really appreciate all of the time you put in to help troubleshoot with me. I'm definitely not a web expert and learned a lot during the process so it was great in a lot of ways. Do you or does the project take donations/crypto? I know this project is going to dramatically increase my productivity so it's the least I could do.

Much appreciated

Cingulara commented 3 years ago

Great! Ok glad to hear it. And I needed to get the HTTPS 100% finalized anyway for installation procedures for the next big version I am working on. So it all worked out well.

For the OSS version, no that is free to the world. My way of giving back since I have used other OSS software for years and years.

The OpenRMF Professional version I am working on for larger companies / organizations / gov't agencies / other gov'ts is a licensed paid and supported version. So if you want to move up to that eventually, you can do it that way. Otherwise, run w/ this and get your time back!!

alphillips-lab commented 3 years ago

I will have to keep an eye out. I will definitely be keeping up with this project and recommending it up the chain as a service when it's available. Thanks again for the good work!

Cingulara commented 3 years ago

Sounds good. Go to https://www.openrmf.io/ and join Slack if you have not so you can keep up with the group.

The Professional version will have a few major upgrades people have been asking for the last 2+ yrs this has been out there. the first major public code commit was Christmas 2018 believe it or not. The highlights are below.

tjmullicani commented 3 years ago

I saw the documentation briefly mentioned self-signed CA’s (such as an Active Directory CA), but I thought it would be good to provide an example. You have to include the full certificate chain, by doing one of the following:

  $ cat rootca.crt intermediateca.crt servercert.crt > servercert_bundle.crt

  - ./ssl/certs/servercert_bundle.crt:/etc/ssl/certs/servercert_bundle.crt:ro

or

  - ./ssl/certs/rootca.crt:/etc/ssl/certs/rootca.crt:ro

  - ./ssl/certs/intermediate.crt:/etc/ssl/certs/intermediateca.crt:ro

  - ./ssl/certs/servercert.crt:/etc/ssl/certs/servercert.crt:ro
Cingulara commented 3 years ago

This is excellent thank you! I need to work on the helm chart this weekend. I will work on updating the documentation for this as well with your example thank you.

Cingulara commented 3 years ago

@tjmullicani I put this into the docs setup in /develop just now. Next release it will be updated for all to see thank you!