search

cryostatio / cryostat

Secure JDK Flight Recorder management for containerized JVMs
https://cryostat.io
Other
19 stars 10 forks source link

[Bug] Discovery Plugin registrations and deregistrations behave badly #406

Closed andrewazores closed 5 months ago

andrewazores commented 6 months ago

Current Behavior

When discovery plugins deregister an exception is thrown and the transaction is not committed. The discovery tree still contains the plugin's realm subtree.

Also, after a discovery plugin registers, it is visible in GET /api/v3/discovery_plugins and its subtree leaves in GET /api/v3/targets, but it does not appear in GET /api/v3/discovery (which also has the effect of making it not visible in the Topology view anymore).

Expected Behavior

When discovery plugins deregister, they should be removed from the discovery tree along with all their targets. Each lost target should generate a WebSocket notification.

Steps To Reproduce

  1. ./smoketest.bash -t
  2. Wait for everything to come up
  3. podman stop compose_quarkus-test-agent_1
  4. Check the Cryostat server logs. Also, the discovery plugin and its target(s) will still appear in API listings and no Discovery WebSocket target loss notification is emitted

Environment

No response

Anything else?

No response

Probably related to #325 due to the new node parent field as well.

andrewazores commented 6 months ago 
db_1                  | 2024-04-24 19:07:33.425 UTC [86] ERROR:  null value in column "realm_id" of relation "discoveryplugin" violates not-null constraint
db_1                  | 2024-04-24 19:07:33.425 UTC [86] DETAIL:  Failing row contains (f, null, e69a1bd8-15da-41c3-86c9-869d2b918d78, http://storedcredentials:4@quarkus-test-agent:9977/).
db_1                  | 2024-04-24 19:07:33.425 UTC [86] STATEMENT:  update DiscoveryPlugin set builtin=$1,realm_id=$2 where id=$3
cryostat_1            | Apr 24, 2024 7:07:33 PM org.hibernate.engine.jdbc.spi.SqlExceptionHelper logExceptions
cryostat_1            | WARN: SQL Error: 0, SQLState: 23502
cryostat_1            | Apr 24, 2024 7:07:33 PM org.hibernate.engine.jdbc.spi.SqlExceptionHelper logExceptions
cryostat_1            | ERROR: ERROR: null value in column "realm_id" of relation "discoveryplugin" violates not-null constraint
cryostat_1            |   Detail: Failing row contains (f, null, e69a1bd8-15da-41c3-86c9-869d2b918d78, http://storedcredentials:4@quarkus-test-agent:9977/).
cryostat_1            | Apr 24, 2024 7:07:33 PM com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator beforeCompletion
cryostat_1            | WARN: ARJUNA012125: TwoPhaseCoordinator.beforeCompletion - failed for SynchronizationImple< 0:ffff0a590a30:96fb:66295763:18a, org.hibernate.resource.transaction.backend.jta.internal.synchronization.RegisteredSynchronization@4cd1c941 >
cryostat_1            | org.hibernate.exception.ConstraintViolationException: could not execute statement [ERROR: null value in column "realm_id" of relation "discoveryplugin" violates not-null constraint
cryostat_1            |   Detail: Failing row contains (f, null, e69a1bd8-15da-41c3-86c9-869d2b918d78, http://storedcredentials:4@quarkus-test-agent:9977/).] [update DiscoveryPlugin set builtin=?,realm_id=? where id=?]
cryostat_1            |     at org.hibernate.exception.internal.SQLStateConversionDelegate.convert(SQLStateConversionDelegate.java:95)
cryostat_1            |     at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:56)
cryostat_1            |     at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:108)
cryostat_1            |     at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:278)
cryostat_1            |     at org.hibernate.engine.jdbc.mutation.internal.AbstractMutationExecutor.performNonBatchedMutation(AbstractMutationExecutor.java:107)
cryostat_1            |     at org.hibernate.engine.jdbc.mutation.internal.MutationExecutorSingleNonBatched.performNonBatchedOperations(MutationExecutorSingleNonBatched.java:40)
cryostat_1            |     at org.hibernate.engine.jdbc.mutation.internal.AbstractMutationExecutor.execute(AbstractMutationExecutor.java:52)
cryostat_1            |     at org.hibernate.persister.entity.mutation.UpdateCoordinatorStandard.doStaticUpdate(UpdateCoordinatorStandard.java:771)
cryostat_1            |     at org.hibernate.persister.entity.mutation.UpdateCoordinatorStandard.performUpdate(UpdateCoordinatorStandard.java:327)
cryostat_1            |     at org.hibernate.persister.entity.mutation.UpdateCoordinatorStandard.coordinateUpdate(UpdateCoordinatorStandard.java:244)
cryostat_1            |     at org.hibernate.persister.entity.AbstractEntityPersister.update(AbstractEntityPersister.java:2749)
cryostat_1            |     at org.hibernate.action.internal.EntityUpdateAction.execute(EntityUpdateAction.java:166)
cryostat_1            |     at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:635)
cryostat_1            |     at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:502)
cryostat_1            |     at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:364)
cryostat_1            |     at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:39)
cryostat_1            |     at org.hibernate.event.service.internal.EventListenerGroupImpl.fireEventOnEachListener(EventListenerGroupImpl.java:127)
cryostat_1            |     at org.hibernate.internal.SessionImpl.doFlush(SessionImpl.java:1412)
cryostat_1            |     at org.hibernate.internal.SessionImpl.managedFlush(SessionImpl.java:485)
cryostat_1            |     at org.hibernate.internal.SessionImpl.flushBeforeTransactionCompletion(SessionImpl.java:2301)
cryostat_1            |     at org.hibernate.internal.SessionImpl.beforeTransactionCompletion(SessionImpl.java:1966)
cryostat_1            |     at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.beforeTransactionCompletion(JdbcCoordinatorImpl.java:439)
cryostat_1            |     at org.hibernate.resource.transaction.backend.jta.internal.JtaTransactionCoordinatorImpl.beforeCompletion(JtaTransactionCoordinatorImpl.java:336)
cryostat_1            |     at org.hibernate.resource.transaction.backend.jta.internal.synchronization.SynchronizationCallbackCoordinatorNonTrackingImpl.beforeCompletion(SynchronizationCallbackCoordinatorNonTrackingImpl.java:47)
cryostat_1            |     at org.hibernate.resource.transaction.backend.jta.internal.synchronization.RegisteredSynchronization.beforeCompletion(RegisteredSynchronization.java:37)
cryostat_1            |     at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76)
cryostat_1            |     at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:360)
cryostat_1            |     at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:91)
cryostat_1            |     at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162)
cryostat_1            |     at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1295)
cryostat_1            |     at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:128)
cryostat_1            |     at io.quarkus.narayana.jta.runtime.NotifyingTransactionManager.commit(NotifyingTransactionManager.java:70)
cryostat_1            |     at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorBase.endTransaction(TransactionalInterceptorBase.java:406)
cryostat_1            |     at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorBase.invokeInOurTx(TransactionalInterceptorBase.java:171)
cryostat_1            |     at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorBase.invokeInOurTx(TransactionalInterceptorBase.java:107)
cryostat_1            |     at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorRequired.doIntercept(TransactionalInterceptorRequired.java:38)
cryostat_1            |     at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorBase.intercept(TransactionalInterceptorBase.java:61)
cryostat_1            |     at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorRequired.intercept(TransactionalInterceptorRequired.java:32)
cryostat_1            |     at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorRequired_Bean.intercept(Unknown Source)
cryostat_1            |     at io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:42)
cryostat_1            |     at io.quarkus.arc.impl.AroundInvokeInvocationContext.perform(AroundInvokeInvocationContext.java:30)
cryostat_1            |     at io.quarkus.arc.impl.InvocationContexts.performAroundInvoke(InvocationContexts.java:27)
cryostat_1            |     at io.cryostat.discovery.Discovery_Subclass.deregister(Unknown Source)
cryostat_1            |     at io.cryostat.discovery.Discovery$quarkusrestinvoker$deregister_27a545d9338e10d484fef9bf00905f02848a1a64.invoke(Unknown Source)
cryostat_1            |     at org.jboss.resteasy.reactive.server.handlers.InvocationHandler.handle(InvocationHandler.java:29)
cryostat_1            |     at io.quarkus.resteasy.reactive.server.runtime.QuarkusResteasyReactiveRequestContext.invokeHandler(QuarkusResteasyReactiveRequestContext.java:141)
cryostat_1            |     at org.jboss.resteasy.reactive.common.core.AbstractResteasyReactiveContext.run(AbstractResteasyReactiveContext.java:145)
cryostat_1            |     at io.quarkus.vertx.core.runtime.VertxCoreRecorder$14.runWith(VertxCoreRecorder.java:576)
cryostat_1            |     at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2513)
cryostat_1            |     at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1538)
cryostat_1            |     at org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:29)
cryostat_1            |     at org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:29)
cryostat_1            |     at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
cryostat_1            |     at java.base/java.lang.Thread.run(Thread.java:840)
cryostat_1            | Caused by: org.postgresql.util.PSQLException: ERROR: null value in column "realm_id" of relation "discoveryplugin" violates not-null constraint
cryostat_1            |   Detail: Failing row contains (f, null, e69a1bd8-15da-41c3-86c9-869d2b918d78, http://storedcredentials:4@quarkus-test-agent:9977/).
cryostat_1            |     at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2713)
cryostat_1            |     at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2401)
cryostat_1            |     at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:368)
cryostat_1            |     at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:498)
cryostat_1            |     at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:415)
cryostat_1            |     at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:190)
cryostat_1            |     at org.postgresql.jdbc.PgPreparedStatement.executeUpdate(PgPreparedStatement.java:152)
cryostat_1            |     at io.agroal.pool.wrapper.PreparedStatementWrapper.executeUpdate(PreparedStatementWrapper.java:88)
cryostat_1            |     at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:275)
cryostat_1            |     ... 50 more
andrewazores commented 5 months ago

Done in #415