search

Hyperfoil / Horreum

Benchmark results repository service
https://horreum.hyperfoil.io/
Apache License 2.0
33 stars 30 forks source link

Evaluate Experiments button missing. #1073

Open whitingjr opened 6 months ago

whitingjr commented 6 months ago

Describe the bug

Loading a Dataset to inspect an Experiment outcomes not loading correctly. The Evaluate Experiments button missing as well.

missing-button-where

The UI displays this message

ui-error

The server shows these exceptions

2024-01-08 10:41:21,235 ERROR [io.qua.ver.htt.run.QuarkusErrorHandler] (executor-thread-115) HTTP Request to /api/experiment/20/profiles failed, error id: 471f2550-e1eb-486e-a9e8-d0ad40988d3c-471: org.hibernate.InstantiationException: Could not instantiate entity : io.hyperfoil.tools.horreum.entity.ExperimentComparisonDAO
    at org.hibernate.metamodel.internal.EmbeddableInstantiatorPojoStandard.instantiate(EmbeddableInstantiatorPojoStandard.java:79)
    at org.hibernate.sql.results.graph.embeddable.AbstractEmbeddableInitializer.createCompositeInstance(AbstractEmbeddableInitializer.java:347)
    at org.hibernate.sql.results.graph.embeddable.AbstractEmbeddableInitializer.prepareCompositeInstance(AbstractEmbeddableInitializer.java:249)
    at org.hibernate.sql.results.graph.embeddable.AbstractEmbeddableInitializer.initializeInstance(AbstractEmbeddableInitializer.java:188)
    at org.hibernate.sql.results.internal.InitializersList.initializeInstance(InitializersList.java:70)
    at org.hibernate.sql.results.internal.StandardRowReader.coordinateInitializers(StandardRowReader.java:111)
    at org.hibernate.sql.results.internal.StandardRowReader.readRow(StandardRowReader.java:87)
    at org.hibernate.sql.results.spi.ListResultsConsumer.consume(ListResultsConsumer.java:179)
    at org.hibernate.sql.results.spi.ListResultsConsumer.consume(ListResultsConsumer.java:33)
    at org.hibernate.sql.exec.internal.JdbcSelectExecutorStandardImpl.doExecuteQuery(JdbcSelectExecutorStandardImpl.java:361)
    at org.hibernate.sql.exec.internal.JdbcSelectExecutorStandardImpl.executeQuery(JdbcSelectExecutorStandardImpl.java:168)
    at org.hibernate.sql.exec.internal.JdbcSelectExecutorStandardImpl.list(JdbcSelectExecutorStandardImpl.java:93)
    at org.hibernate.sql.exec.spi.JdbcSelectExecutor.list(JdbcSelectExecutor.java:31)
    at org.hibernate.loader.ast.internal.CollectionBatchLoaderArrayParam.initializeKeys(CollectionBatchLoaderArrayParam.java:206)
    at org.hibernate.loader.ast.internal.CollectionBatchLoaderArrayParam.load(CollectionBatchLoaderArrayParam.java:126)
    at org.hibernate.persister.collection.AbstractCollectionPersister.initialize(AbstractCollectionPersister.java:680)
    at org.hibernate.event.internal.DefaultInitializeCollectionEventListener.onInitializeCollection(DefaultInitializeCollectionEventListener.java:75)
    at org.hibernate.event.service.internal.EventListenerGroupImpl.fireEventOnEachListener(EventListenerGroupImpl.java:127)
    at org.hibernate.internal.SessionImpl.initializeCollection(SessionImpl.java:1702)
    at org.hibernate.collection.spi.AbstractPersistentCollection.lambda$initialize$3(AbstractPersistentCollection.java:617)
    at org.hibernate.collection.spi.AbstractPersistentCollection.withTemporarySessionIfNeeded(AbstractPersistentCollection.java:265)
    at org.hibernate.collection.spi.AbstractPersistentCollection.initialize(AbstractPersistentCollection.java:615)
    at org.hibernate.collection.spi.AbstractPersistentCollection.forceInitialization(AbstractPersistentCollection.java:813)
    at org.hibernate.engine.internal.StatefulPersistenceContext.initializeNonLazyCollections(StatefulPersistenceContext.java:985)
    at org.hibernate.engine.internal.StatefulPersistenceContext.initializeNonLazyCollections(StatefulPersistenceContext.java:971)
    at org.hibernate.sql.results.spi.ListResultsConsumer.consume(ListResultsConsumer.java:227)
    at org.hibernate.sql.results.spi.ListResultsConsumer.consume(ListResultsConsumer.java:33)
    at org.hibernate.sql.exec.internal.JdbcSelectExecutorStandardImpl.doExecuteQuery(JdbcSelectExecutorStandardImpl.java:361)
    at org.hibernate.sql.exec.internal.JdbcSelectExecutorStandardImpl.executeQuery(JdbcSelectExecutorStandardImpl.java:168)
    at org.hibernate.sql.exec.internal.JdbcSelectExecutorStandardImpl.list(JdbcSelectExecutorStandardImpl.java:93)
    at org.hibernate.sql.exec.spi.JdbcSelectExecutor.list(JdbcSelectExecutor.java:31)
    at org.hibernate.query.sqm.internal.ConcreteSqmSelectQueryPlan.lambda$new$0(ConcreteSqmSelectQueryPlan.java:110)
    at org.hibernate.query.sqm.internal.ConcreteSqmSelectQueryPlan.withCacheableSqmInterpretation(ConcreteSqmSelectQueryPlan.java:303)
    at org.hibernate.query.sqm.internal.ConcreteSqmSelectQueryPlan.performList(ConcreteSqmSelectQueryPlan.java:244)
    at org.hibernate.query.sqm.internal.QuerySqmImpl.doList(QuerySqmImpl.java:518)
    at org.hibernate.query.spi.AbstractSelectionQuery.list(AbstractSelectionQuery.java:367)
    at org.hibernate.query.Query.getResultList(Query.java:119)
    at io.quarkus.hibernate.orm.panache.common.runtime.CommonPanacheQueryImpl.list(CommonPanacheQueryImpl.java:280)
    at io.quarkus.hibernate.orm.panache.runtime.PanacheQueryImpl.list(PanacheQueryImpl.java:149)
    at io.quarkus.hibernate.orm.panache.runtime.JpaOperations.list(JpaOperations.java:24)
    at io.quarkus.hibernate.orm.panache.runtime.JpaOperations.list(JpaOperations.java:10)
    at io.quarkus.hibernate.orm.panache.common.runtime.AbstractJpaOperations.list(AbstractJpaOperations.java:245)
    at io.hyperfoil.tools.horreum.entity.ExperimentProfileDAO.list(ExperimentProfileDAO.java)
    at io.hyperfoil.tools.horreum.svc.ExperimentServiceImpl.profiles(ExperimentServiceImpl.java:75)
    at io.hyperfoil.tools.horreum.svc.ExperimentServiceImpl_Subclass.profiles$$superforward(Unknown Source)
    at io.hyperfoil.tools.horreum.svc.ExperimentServiceImpl_Subclass$$function$$2.apply(Unknown Source)
    at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:73)
    at io.quarkus.arc.impl.AroundInvokeInvocationContext$NextAroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:97)
    at io.hyperfoil.tools.horreum.server.RolesInterceptor.intercept(RolesInterceptor.java:53)
    at io.hyperfoil.tools.horreum.server.RolesInterceptor_Bean.intercept(Unknown Source)
    at io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:42)
    at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:70)
    at io.quarkus.arc.impl.AroundInvokeInvocationContext$NextAroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:97)
    at io.quarkus.security.runtime.interceptor.SecurityHandler.handle(SecurityHandler.java:47)
    at io.quarkus.security.runtime.interceptor.PermitAllInterceptor.intercept(PermitAllInterceptor.java:23)
    at io.quarkus.security.runtime.interceptor.PermitAllInterceptor_Bean.intercept(Unknown Source)
    at io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:42)
    at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:70)
    at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:62)
    at io.quarkus.resteasy.reactive.server.runtime.StandardSecurityCheckInterceptor.intercept(StandardSecurityCheckInterceptor.java:44)
    at io.quarkus.resteasy.reactive.server.runtime.StandardSecurityCheckInterceptor_PermitAllInterceptor_Bean.intercept(Unknown Source)
    at io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:42)
    at io.quarkus.arc.impl.AroundInvokeInvocationContext.perform(AroundInvokeInvocationContext.java:30)
    at io.quarkus.arc.impl.InvocationContexts.performAroundInvoke(InvocationContexts.java:27)
    at io.hyperfoil.tools.horreum.svc.ExperimentServiceImpl_Subclass.profiles(Unknown Source)
    at io.hyperfoil.tools.horreum.svc.ExperimentServiceImpl_ClientProxy.profiles(Unknown Source)
    at io.hyperfoil.tools.horreum.api.services.ExperimentService$quarkusrestinvoker$profiles_b36d63b9a4ce2caa88bc9b0f6654ee51de984e63.invoke(Unknown Source)
    at org.jboss.resteasy.reactive.server.handlers.InvocationHandler.handle(InvocationHandler.java:29)
    at io.quarkus.resteasy.reactive.server.runtime.QuarkusResteasyReactiveRequestContext.invokeHandler(QuarkusResteasyReactiveRequestContext.java:141)
    at org.jboss.resteasy.reactive.common.core.AbstractResteasyReactiveContext.run(AbstractResteasyReactiveContext.java:147)
    at io.quarkus.vertx.core.runtime.VertxCoreRecorder$14.runWith(VertxCoreRecorder.java:582)
    at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2513)
    at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1512)
    at org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:29)
    at org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:29)
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: org.hibernate.PropertyAccessException: Could not set value of type [com.fasterxml.jackson.databind.node.TextNode] : `io.hyperfoil.tools.horreum.entity.ExperimentComparisonDAO.config` (setter)
    at org.hibernate.property.access.spi.SetterFieldImpl.set(SetterFieldImpl.java:86)
    at org.hibernate.property.access.spi.EnhancedSetterImpl.set(EnhancedSetterImpl.java:40)
    at org.hibernate.metamodel.mapping.internal.AbstractEmbeddableMapping.setValues(AbstractEmbeddableMapping.java:112)
    at org.hibernate.metamodel.internal.EmbeddableInstantiatorPojoStandard.instantiate(EmbeddableInstantiatorPojoStandard.java:73)
    ... 76 more
Caused by: java.lang.IllegalArgumentException: Can not set com.fasterxml.jackson.databind.node.ObjectNode field io.hyperfoil.tools.horreum.entity.ExperimentComparisonDAO.config to com.fasterxml.jackson.databind.node.TextNode
    at java.base/jdk.internal.reflect.UnsafeFieldAccessorImpl.throwSetIllegalArgumentException(UnsafeFieldAccessorImpl.java:167)
    at java.base/jdk.internal.reflect.UnsafeFieldAccessorImpl.throwSetIllegalArgumentException(UnsafeFieldAccessorImpl.java:171)
    at java.base/jdk.internal.reflect.UnsafeObjectFieldAccessorImpl.set(UnsafeObjectFieldAccessorImpl.java:81)
    at java.base/java.lang.reflect.Field.set(Field.java:799)
    at org.hibernate.property.access.spi.SetterFieldImpl.set(SetterFieldImpl.java:55)
    ... 79 more

This is seen using this path /run/23634#dataset0 on the system I use.

The cause of this error is likely due to missing data for particular runs. This happens when developing the test by adding new data and Labels.

Version

0.10.7
whitingjr commented 6 months ago

This is due to orphaned data in the db. The persisted json format occured when 0.10.4 was deployed. Now 0.10.7 is deployed the format can no longer be loaded. I will re-create my Experiment profile.

whitingjr commented 6 months ago

This is the record in experiment_profiles table. The runs being processed seem to cause the experiment service to fail. 

   id    |              name              | test_id |       selector_labels        |          selector_filter          |      baseline_labels       |          baseline_filter          | extra_labels ---------+--------------------------------+---------+------------------------------+-----------------------------------+----------------------------+-----------------------------------+--------------
 5363149 | some_comparison                |      20 | ["foo"]                      | value => {return value === 2500;} | ["bar"]                    | value => {return value === 1888;} |
johnaohara commented 6 months ago

@whitingjr wrt the "Experiments" button missing, that button is hidden when there are no experiments defined. It would probably be better to grey out the button instead. Please can you open a separate issue for that?

whitingjr commented 6 months ago

The exception in this issue is also breaking export Test functionality for my Test. Other Tests are working as expected. Same error seen in the server log. export-button-click-error

whitingjr commented 6 months ago

@whitingjr wrt the "Experiments" button missing, that button is hidden when there are no experiments defined. It would probably be better to grey out the button instead. Please can you open a separate issue for that? @johnaohara #1077

johnaohara commented 6 months ago

@whitingjr has the issue with the data been resolved, i.e. that lead to : Caused by: java.lang.IllegalArgumentException: Can not set com.fasterxml.jackson.databind.node.ObjectNode field io.hyperfoil.tools.horreum.entity.ExperimentComparisonDAO.config to com.fasterxml.jackson.databind.node.TextNode

Was there a PR opened to fix that?

johnaohara commented 6 months ago

I am re-opening the issue so that we do not lose track of this problem, if there is a PR that fixes it, please close with a reference to that PR

whitingjr commented 6 months ago

@johnaohara #1122

johnaohara commented 6 months ago

@whitingjr pr #1122 related to https://github.com/Hyperfoil/Horreum/issues/1077

This issue is a different issue

whitingjr commented 6 months ago

@johnaohara A fix was not added. Discussed elsewhere in a meeting. It was decided by @stalep and @johnaohara no code fix should be added. Instead the Experiment configuration was re-created.

johnaohara commented 6 months ago

I recollect that we needed to resolve the data issue in the database. A version populated the database with corrupt data, which needs resolving either by;

1 - A data fix applied in changelog.xml or 2 - Documentation of what the issue is and how to resolve it manually.

If a user experiences this problem, how do they resolve it?

johnaohara commented 6 months ago

@whitingjr my preference would be (1) ^

whitingjr commented 6 months ago

@johnaohara is there any point now that the data has already been re-created and we are unlikely to be rolling back Horreum that used the different type mapping ?

johnaohara commented 6 months ago

@johnaohara is there any point now that the data has already been re-created and we are unlikely to be rolling back Horreum that used the different type mapping ?

You are referring to a workaround implemented in one specific instance of Horreum running within one company

For users of this project, outside of our team, outside of Red Hat, how do they resolve the same problem?

whitingjr commented 5 months ago

In that case search for the row in the db. I then deleted the row. Then using the UI created the deleted Experiment Profile.

johnaohara commented 5 months ago

In that case search for the row in the db. I then deleted the row. Then using the UI created the deleted Experiment Profile.

That is the workaround that you used, but I would not expect users to have to do that.