search

p2-inc / keycloak-orgs

Single realm, multi-tenancy for SaaS apps
https://phasetwo.io
Other
389 stars 66 forks source link

Better documentation/how to use #10

Closed KDMRyanT closed 1 year ago

KDMRyanT commented 2 years ago

Hello, I posted earlier, how to use orgs - its through restapi only.

Now I'm looking for how to setup and utilize - I have created org(s) within a realm. Looking for a full run through or use case.

xgp commented 2 years ago

Hi @KDMRyanT There are lots of ways to us this. Can you tell me what you are trying to achieve?

KDMRyanT commented 2 years ago

Hello, what I'm trying to achieve is multi tenancy for application with a single keycloak.json install file for the client. Then need to have multiple IdP saml sites setup and create map for each member to specific organization. Further, once that is confirmed to be working, set this all up behind a proxy.

I have so far been able to create Orgs that are related in name to the IdP sites. I don't know what to place into that from the api document. I have also even on the basic stuff trying to find what is the purpose of "url", domains, and attributes of the org object?

KDMRyanT commented 2 years ago

Another question, can there be 1 realm for say the application clients and then other realms for each of the IdPs. Then in the application client realm this is where you use Orgs and in creating the org - your specifying the correct realm? Right now, I have been creating all the object under 1 realm called testshib and in creating a org in that realm, I also say realm = testshib. Is that right?

xgp commented 2 years ago

Thanks for the description. I can write up something that will help describe the facilities available for your use case.

On Fri, Aug 19, 2022, at 7:28 AM, KDMRyanT wrote:

Another question, can there be 1 realm for say the application clients and then other realms for each of the IdPs. Then in the application client realm this is where you use Orgs and in creating the org - your specifying the correct realm? Right now, I have been creating all the object under 1 realm called testshib and in creating a org in that realm, I also say realm = testshib. Is that right?

— Reply to this email directly, view it on GitHub https://github.com/p2-inc/keycloak-orgs/issues/10#issuecomment-1220745770, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAB3UHK6MKXNKOJCHZNOH3DVZ6KY7ANCNFSM56XITBYA. You are receiving this because you commented.Message ID: @.***>

KDMRyanT commented 1 year ago

Hi @xgp Did some more work in my keycloak environment. Did some setup of organizations and mapped idp users added to orgs created. Added to client a openid-connect mapper of type Organization Role and added json to token claim called organizations. Next I hit the client I setup, logged in on one of the idps I had setup under realm and now in the id token and userinfo output I have organizations! I do have a screenshot of that add in keycloak if you want. Still looking more though towards this issue. I'm definitely not utilizing the full power of this extension or its full abilities.

Cheers

xgp commented 1 year ago

organization-role-oidc-mapper-in-keycloak

Thanks @KDMRyanT ! I'll add this to the docs when I get more time. Let's keep this issue open for now.

Also, I'm working on publishing a project that has a decent example of how to use organizations and their roles, invitations and SSO/IdP setup. I'll post here once I clean that repo up and switch it to public.

KDMRyanT commented 1 year ago

Hello @xgp, Its been a while since I have connected with you. I'm CDN so just got back from thanksgiving holidays! Anyways, wanted to get back to you on this subject. Above you reference "a decent example of how to use organizations and their roles, invitations and SSO/IdP setup" and perhaps its in a separate repo? Is that ready?

In doing some searching, I have noticed as well that p2inc has a containers repository and methods to setup a deployment using p2inc code like keycloak-orgs. Is this what your referring to? Should this be the path or a path to figuring out? Also referenced there is "documentation/examples"? Let me know

Thanks @KDMRyanT

xgp commented 1 year ago

Hi @KDMRyanT - Hope your holiday was good. A few things:

janhaesen commented 1 year ago

Hi @xgp ,

based of your post on the Keycloak discourse group I found this extension which, like for many others, seems to address exactly the issue that I'm trying to resolve. So first of, thanks for open-sourcing this extension. Given this ticket I believe that the problems I'm having stems from my misunderstanding and interpretation of documentation. Hence I'd comment here on what I'm encountering, where you perhaps can shed some light on it.

I've gone through the installation steps by cloning the Keycloak repo and this repo. Build Keycloak and package and, as I understood it, copy the package/jar created in the target dir to a new folder, called providers, in the root of Keycloak. Upon starting Keycloak I did not get the idea that the provider was in fact present, or perhaps I am simply just missing something, which is highly likely. Do I understand correctly that in order to have the UI reflect organisations I need to have https://github.com/p2-inc/keycloak-ui present as well?

Based of the documentation and your comment here I also tried to run the docker image that was already on quay, but I was unable to get that running properly. I have a docker compose file that starts a postgres database and starts the container with the Keycloak image to be stalling on database migration:

2022-11-10 09:51:56,145 INFO  [org.keycloak.quarkus.runtime.storage.legacy.liquibase.QuarkusJpaUpdaterProvider] (main) Initializing database schema. Using changelog META-INF/jpa-changelog-master.xml

is that something you've seen before?

I also tried:

docker run --name phasetwo_test -p 8080:8080 \
    -e KEYCLOAK_ADMIN=admin -e KEYCLOAK_ADMIN_PASSWORD=admin -e KC_HTTP_RELATIVE_PATH=/auth \
    quay.io/phasetwo/phasetwo-keycloak:latest \
    start-dev

where I replaced latest with the version 20 alpha, and that did not come any further than 1 logline: "Updating the configuration and installing your custom providers, if any. Please wait.". I did get a warning that the image platform and my platform did not match (I'm on Apple M1).

In summary:

  1. Is the providers dir in the root of the cloned Keycloak repo correct, if not where should it go? I've found that the dir in question is there when using the distribution directly from Keycloak. I however don't seem to get/have this when building from source, likely I'm missing a step or something isn't working.
  2. I take it I need the keycloak-ui repo as well in order to get the admin UI to present me with organizations, or is that not yet available?
  3. Should Keycloak be rebuild once the provider is added before starting it, what would be the correct command to get it started

Cheers!

P.S. something I noticed during the packaging of the extension is there's a conflict in the DNS Record class used. The import done is a asterisk, while if you replace it with: org.xbill.DNS.Record it will work. I would generally advise against import all statements. I encountered this when on java 17. Full message I got was:

[86,21] error: reference to Record is ambiguous
[ERROR]   both class org.xbill.DNS.Record in org.xbill.DNS and class java.lang.Record in java.lang match
xgp commented 1 year ago

Hi @janhaesen. Thanks for taking a look.

I'm on Apple M1

It won't work on M1.

Is the providers dir in the root of the cloned Keycloak repo correct, if not where should it go?

Sorry, I don't understand this. Can you be more specific as to which directories and which repos you are referring to?

I take it I need the keycloak-ui repo as well in order to get the admin UI to present me with organizations, or is that not yet available?

Yes, that's where the admin ui code lives. While we're getting 20.x working, it lives in the 20.0.0_orgs branch https://github.com/p2-inc/keycloak-ui/tree/20.0.0_orgs

Should Keycloak be rebuild once the provider is added before starting it, what would be the correct command to get it started

If you add an extension to the providers dir in keycloak, you should rebuild it. Docs on how to re/build are here https://www.keycloak.org/server/configuration

DNS Record class

Thanks. I'll hadn't seen this conflict before.

janhaesen commented 1 year ago

Thanks for getting back to me.

Hi @janhaesen. Thanks for taking a look.

I'm on Apple M1

It won't work on M1.

Good to know, I will come up with a setup that will most likely do it through Docker to circumvent this.

Is the providers dir in the root of the cloned Keycloak repo correct, if not where should it go?

Sorry, I don't understand this. Can you be more specific as to which directories and which repos you are referring to?

I'm referring to Keycloak itself, after having checked out version 20.0.1 I did a mvn install -Pdistribution -DskipTests. After which I need to place the package that was bundled for keycloak-orgs (this repo) in the distribution of Keycloak, based of the readme in this repo. It's however unclear exactly where, perhaps due to the newer version, but I don't seem to get a distribution as when I download it directly from Keycloak. I however take it that this might also be described in your referred Keycloak documentation (https://www.keycloak.org/server/configuration), so I will check that out.

xgp commented 1 year ago

You don't need to build keycloak itself to use this. Either using a bundle (tar.gz or zip from https://www.keycloak.org/downloads) or the Docker image, put the keycloak-orgs jar in the providers/ directory at the top level of the keycloak distribution (e.g. /opt/keycloak/providers/ in the case of the Docker image).

However, I'd suggest just using the Docker image that has this and the UI bundled so you don't need to put it together yourself. https://quay.io/repository/phasetwo/phasetwo-keycloak?tab=tags

janhaesen commented 1 year ago

Concerning what I'm trying to achieve (get a running instance to see the organisations at work). I've been able to get an external postgres container running with a keycloak container that connects to it, I was messing about with this for a bit but we're now good to go. The database migration is working well and I see organization related tables present in the database. Based of your information I would assume that the organization management would be in the UI of Keycloak admin panel, but I'm unable to find it. I see that the resources for organization are behind a role so I went ahead and assigned the admin user these roles, but I'm not yet able to see anything pop up in the menu. Is that expected? Reason I'm asking is that I'd like this to be operationally scalable and doing this kind of management purely API based will prevent that. It's worth noting that I found that I need to set the theme, which I've done.

On some other notes, perhaps good for documentation

  1. I've tried the latest container that was available and was able to get it running on an Apple M1 with: 
    docker run --name phasetwo_test -p 8080:8080 \
--platform linux/amd64 \
-e KEYCLOAK_ADMIN=admin -e KEYCLOAK_ADMIN_PASSWORD=admin -e KC_HTTP_RELATIVE_PATH=/auth \
quay.io/phasetwo/phasetwo-keycloak:20.0.0-alpha \
start-dev
  2. Next to that I was wondering, based of the documentation you're suggesting that you can also run it based of Keycloak source, that wasn't possible for me with the existing commands provided in the documentation. Could be worth updating/amending to make that work correctly with perhaps some more explicit steps. Especially considering the Quarkus and Wildfly options you have with Keycloak.
xgp commented 1 year ago

@janhaesen Keycloak requires you to set the admin "theme" in order to access the custom Organization UI. Once you log in to your Keycloak Admin UI, select Realm Settings -> Themes and change the admin theme to phasetwo.v2. You have to log out and log back in to use the new theme.

Better documentation on how to run this from your own Keycloak instance is not present, as it's assumed that you are familiar with how Keycloak extensions are built and installed if you are going to DIY. If you are encountering problems, feel free to post here. There is a good list of jars that need to go in the providers/ dir (Quarkus distro) in the repo where our docker images get built https://github.com/p2-inc/phasetwo-containers/tree/main/lib . You can throw out keycloak-events, keycloak-magic-link and keycloak-rest-provider, but the others need to be there.

phamann commented 1 year ago

Also, I'm working on publishing a project that has a decent example of how to use organizations and their roles, invitations and SSO/IdP setup. I'll post here once I clean that repo up and switch it to public.

Hey @xgp, did you ever manage to make your example repo public? I'm currently evaluating the extension and would love to see a more thorough example. Thanks

sundbp commented 1 year ago

I'm looking at this part of the docs: https://phasetwo.io/docs/organizations/identity-providers

Under "Verified Domains" it says:

The purpose of associating domains with an organization is to allow identity providers to be conditionally shown based on email domain of a user who is logging in. In the Settings tab of each organizations, you can add multiple domains. When an authentication flow is configured properly to allow for redirection based on email domain, these are the domains that will be used to look up the associating identity provider.

For the part I've bolded - what does "configured properly" look like in this case? I'd love an example!

Thanks!

sundbp commented 1 year ago

Read the code and believe it's something like this: image

xgp commented 1 year ago

@phamann We are still in the process of open sourcing the "Admin portal" that will contain the best example of the use of the orgs APIs. In the meantime, a good place to look if you are interested in frontend code is our extension to the Keycloak Admin UI that includes configuration for all aspects of an organization. https://github.com/p2-inc/keycloak-ui/tree/20.0.1_orgs/apps/admin-ui/src/orgs

xgp commented 1 year ago

Closing this, as it's becoming a catch-all. Please file specific bugs and feature requests.

xgp commented 1 year ago

FYI to those who asked here. We open sourced our "Admin Portal" that allows users to self-manage their profile and organizations. It would be a good example of how to use the organizations APIs in a frontend application for anyone who is looking. It also probably covers 80% of the use cases for exposing organization functionality to the user in the UI, so if you're looking for an easy console to replace the Keycloak Account Console + Organization management functionality, this is it:

https://github.com/p2-inc/phasetwo-admin-portal