search

keycloak / keycloak

Open Source Identity and Access Management For Modern Applications and Services
https://www.keycloak.org
Apache License 2.0
23.72k stars 6.81k forks source link

Migrating from keycloak 15 to keycloak 20.0.1 #16074

Open ajaiswal595 opened 1 year ago

ajaiswal595 commented 1 year ago

Description

I'm upgrading from 15 to 20.0.1, getting below error.

I'm trying to use the existing database used by keycloak 15.

Initially is throws error regrading changelog check-sum. So i added this in changelog file.

7:53188c3eb1107546e6f765835705b6c1

Now I'm getting this error and have no clue what is breaking and how to fix it.

2022-12-19 19:18:35,076 INFO [org.infinispan.CLUSTER] (main) ISPN000080: Disconnecting JGroups channel ISPN 2022-12-19 19:18:35,134 ERROR [org.keycloak.quarkus.runtime.cli.ExecutionExceptionHandler] (main) ERROR: Failed to start server in (production) mode 2022-12-19 19:18:35,134 ERROR [org.keycloak.quarkus.runtime.cli.ExecutionExceptionHandler] (main) Error details:: java.lang.NullPointerException at org.keycloak.migration.migrators.MigrateTo20_0_0.addViewGroupsRole(MigrateTo20_0_0.java:33) at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195) at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195) at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177) at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195) at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133) at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801) at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497) at org.hibernate.query.spi.StreamDecorator.forEach(StreamDecorator.java:153) at org.keycloak.utils.ClosingStream.forEach(ClosingStream.java:128) at org.keycloak.migration.migrators.MigrateTo20_0_0.migrate(MigrateTo20_0_0.java:19) at org.keycloak.storage.datastore.LegacyMigrationManager.migrate(LegacyMigrationManager.java:133) at org.keycloak.migration.MigrationModelManager.migrate(MigrationModelManager.java:33) at org.keycloak.quarkus.runtime.storage.legacy.database.LegacyJpaConnectionProviderFactory.migrateModel(LegacyJpaConnectionProviderFactory.java:219) at org.keycloak.quarkus.runtime.storage.legacy.database.LegacyJpaConnectionProviderFactory.initSchema(LegacyJpaConnectionProviderFactory.java:213) at org.keycloak.models.utils.KeycloakModelUtils.runJobInTransaction(KeycloakModelUtils.java:260) at org.keycloak.quarkus.runtime.storage.legacy.database.LegacyJpaConnectionProviderFactory.postInit(LegacyJpaConnectionProviderFactory.java:138) at org.keycloak.quarkus.runtime.integration.QuarkusKeycloakSessionFactory.init(QuarkusKeycloakSessionFactory.java:104) at org.keycloak.quarkus.runtime.integration.jaxrs.QuarkusKeycloakApplication.startup(QuarkusKeycloakApplication.java:42) at org.keycloak.quarkus.runtime.integration.QuarkusLifecycleObserver.onStartupEvent(QuarkusLifecycleObserver.java:37) at org.keycloak.quarkus.runtime.integration.QuarkusLifecycleObserver_Observer_onStartupEvent_b0e82415b143738dc1f986a5fa4668e83d0a5dea.notify(Unknown Source) at io.quarkus.arc.impl.EventImpl$Notifier.notifyObservers(EventImpl.java:326) at io.quarkus.arc.impl.EventImpl$Notifier.notify(EventImpl.java:308) at io.quarkus.arc.impl.EventImpl.fire(EventImpl.java:76) at io.quarkus.arc.runtime.ArcRecorder.fireLifecycleEvent(ArcRecorder.java:131) at io.quarkus.arc.runtime.ArcRecorder.handleLifecycleEvents(ArcRecorder.java:100) at io.quarkus.deployment.steps.LifecycleEventsBuildStep$startupEvent1144526294.deploy_0(Unknown Source) at io.quarkus.deployment.steps.LifecycleEventsBuildStep$startupEvent1144526294.deploy(Unknown Source) at io.quarkus.runner.ApplicationImpl.doStart(Unknown Source) at io.quarkus.runtime.Application.start(Application.java:101) at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:110) at io.quarkus.runtime.Quarkus.run(Quarkus.java:69) at org.keycloak.quarkus.runtime.KeycloakMain.start(KeycloakMain.java:103) at org.keycloak.quarkus.runtime.cli.command.AbstractStartCommand.run(AbstractStartCommand.java:37) at picocli.CommandLine.executeUserObject(CommandLine.java:1939) at picocli.CommandLine.access$1300(CommandLine.java:145) at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2358) at picocli.CommandLine$RunLast.handle(CommandLine.java:2352) at picocli.CommandLine$RunLast.handle(CommandLine.java:2314) at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179) at picocli.CommandLine$RunLast.execute(CommandLine.java:2316) at picocli.CommandLine.execute(CommandLine.java:2078) at org.keycloak.quarkus.runtime.cli.Picocli.parseAndRun(Picocli.java:93) at org.keycloak.quarkus.runtime.KeycloakMain.main(KeycloakMain.java:89) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at io.quarkus.bootstrap.runner.QuarkusEntryPoint.doRun(QuarkusEntryPoint.java:61) at io.quarkus.bootstrap.runner.QuarkusEntryPoint.main(QuarkusEntryPoint.java:32)

Discussion

https://keycloak.discourse.group/t/migration-fail-when-upgrading-from-v19-0-1-to-v20-0-1/18496/4

Issues

It should be a smooth upgrade from Keyclaok 15 to Keycloak 20.

Motivation

No response

lexcao commented 1 year ago

Hi, could you please check these issues?

It seems you have deleted account-console client from your realm.

The solution is to add it back manually.

pedroigor commented 1 year ago

@ajaiswal595 Looks like you closed https://github.com/keycloak/keycloak/issues/15826. Can we close this one too?

ajaiswal595 commented 1 year ago

@ajaiswal595 Looks like you closed #15826. Can we close this one too?

Here is my analysis and ask.

What I found is while (migrating) pointing to the Keycloak legacy(Keycloak 15) database with Keycloak 20.0.1 it is looking for an entry "account-console" in the client Table.

So I add that record manually in the database. Then started the keycloak.

Then i got this error

2022-12-20 18:52:01,038 ERROR [org.keycloak.quarkus.runtime.cli.ExecutionExceptionHandler] (main) ERROR: Failed to start server in (production) mode 2022-12-20 18:52:01,038 ERROR [org.keycloak.quarkus.runtime.cli.ExecutionExceptionHandler] (main) Error details:: java.lang.RuntimeException: Script upload is disabled at org.keycloak.authorization.policy.provider.js.JSPolicyProviderFactory.throwCanNotUpdatePolicy(JSPolicyProviderFactory.java:130)

To resolve this error i had to find default-policy in Database and deleted record from table "resource-server-policy" , "policy-config" and associated-policy.

After that i could able to Start Keycloak Quarkus with Legacy Database.

So one question i have

Is there a fix for the above manual process?

pedroigor commented 1 year ago

I see now.

I don't think we are going to fix those issues, but I'm not sure.

For instance, the last error is about not being able anymore to manage providers that rely on JS code. Instead, you should deploy a JAR file and configure these providers properly as per https://www.keycloak.org/docs/latest/server_development/#_script_providers. I'm not sure how we can improve migration for this case without end-up in a non-working realm as some providers won't be fully configured.

@stianst wdyt?

stianst commented 1 year ago

@pedroigor Perhaps we should delete the JS policies, and print warning logs that they have been deleted? Or, at least we should document somewhere what folks should do prior to upgrading (without asking them to delete things directly from the DB of course).

pedroigor commented 1 year ago

@stianst @ajaiswal595 I'm not sure if we should delete data without any approval from the admin. For me, looks like the best to do here is to update our docs.

ajaiswal595 commented 1 year ago

@pedroigor I'm in the process of migration.

And these two things are making migration tough.

1) We need to put an account-console entry as one client in the legacy database manually. 2) For the default policy of type js, also we have to update the record in the resource-server-policy table

Is there any timeline you can suggest to me so I can keep its hold for some time?

As of now, I'm trying to do this database operation using liquibase changelog.

pedroigor commented 1 year ago

@ajaiswal595 I'm not sure if #1 above is an issue as per the last comments from here https://github.com/keycloak/keycloak/issues/15738#issuecomment-1361067525.

W.r.t. #2, the best we can do IMO is to change docs to say that "Before migration, make sure to update your JS policies to reference JS policy providers deployed to the server.". Something along these lines.