aruvic
commented
2 weeks ago Open aruvic opened 2 weeks ago
aruvic
commented
2 weeks ago 
arcuri82
commented
2 weeks ago @aruvic thanks for reporting this bug. For now, can you try to run with --doCollectImpact false --probOfArchiveMutation 0 to disable that part of the code?
@man-zhang do you think you have some time to look into this issue?
aruvic
commented
2 weeks ago Thanks, it worked now. Please can you just check if the current behaviour is as expected.
Driver was run as following. Can I safely ignore the warnings?
WARN - Fail to load org.evomaster.client.java.instrumentation.staticstate.ExecutionTracer
21:56:53.524 [main] INFO shaded.org.eclipse.jetty.util.log -- Logging initialized @2068ms to shaded.org.eclipse.jetty.util.log.Slf4jLog
21:56:53.723 [main] WARN shaded.org.eclipse.jetty.server.handler.ContextHandler -- s.o.e.j.s.ServletContextHandler@4a8355dd{/,null,UNAVAILABLE} contextPath ends with /*
21:56:53.737 [main] INFO shaded.org.eclipse.jetty.server.Server -- jetty-9.4.29.v20200521; built: 2020-05-21T17:20:40.598Z; git: 77c232aed8a45c818fd27232278d9f95a021095e; jvm 17.0.11+9
May 15, 2024 9:56:54 PM shaded.org.glassfish.jersey.server.wadl.WadlFeature configure
WARNING: JAXBContext implementation could not be found. WADL feature is disabled.
May 15, 2024 9:56:54 PM shaded.org.glassfish.jersey.internal.inject.Providers checkProviderRuntime
WARNING: A provider org.evomaster.client.java.controller.internal.EMController registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider org.evomaster.client.java.controller.internal.EMController will be ignored.
May 15, 2024 9:56:55 PM shaded.org.glassfish.jersey.internal.Errors logErrors
WARNING: The following warnings have been detected: WARNING: The (sub)resource method getWarning in org.evomaster.client.java.controller.internal.EMController contains empty path annotation.
21:56:55.092 [main] INFO shaded.org.eclipse.jetty.server.handler.ContextHandler -- Started s.o.e.j.s.ServletContextHandler@4a8355dd{/controller/api,null,AVAILABLE}
21:56:55.173 [main] INFO shaded.org.eclipse.jetty.server.AbstractConnector -- Started ServerConnector@3e27aa33{HTTP/1.1, (http/1.1)}{localhost:40100}
21:56:55.173 [main] INFO shaded.org.eclipse.jetty.server.Server -- Started @3735ms
Started controller server on: http://localhost:40100/controller/apiAfter Driver was running EvoMaster with the additional flags was working. Please can you check if the output of the run is as expected? Needed budget: 26%, Does this mean I can run this in 26% of 10 minutes and it will achieve same results? Is Bytecode line coverage: 3% also expected or this is very low?
java --add-opens java.base/java.net=ALL-UNNAMED --add-opens java.base/java.util=ALL-UNNAMED --add-opens java.base/java.util.regex=ALL-UNNAMED --add-opens java.base/java.lang=ALL-UNNAMED -Djdk.attach.allowAttachSelf=true -jar evomaster.jar --outputFolder "C:\Users\aruvic\Documents\Development\new_folder\UC001\JAVA\tmf_api_tmf632_party_management_v5.0.0_server\test_output" --maxTime 10m --doCollectImpact false --probOfArchiveMutation 0
* ←[34m
_____ ___ ___ _
| ___| | \/ | | |
| |____ _____ | . . | __ _ ___| |_ ___ _ __
| __\ \ / / _ \| |\/| |/ _` / __| __/ _ \ '__|
| |___\ V / (_) | | | | (_| \__ \ || __/ |
\____/ \_/ \___/\_| |_/\__,_|___/\__\___|_|
←[0m
* EvoMaster version: 3.0.1-SNAPSHOT
* Loading configuration file from: C:\Users\aruvic\Documents\Development\EvoMaster\core\target\em.yaml
* Initializing...
21:58:50.334 [main] WARN o.t.u.TestcontainersConfiguration - Attempted to read Testcontainers configuration file at file:/C:/Users/aruvic/.testcontainers.properties but the file was not found. Exception message: FileNotFoundException: C:\Users\aruvic\.testcontainers.properties (The system cannot find the file specified)
21:58:52.632 [main] WARN o.e.c.p.rest.RestActionBuilderV3 - Unhandled format 'base64'
21:58:52.635 [main] WARN o.e.c.p.rest.RestActionBuilderV3 - No fields for object definition: partyOrPartyRole
21:58:52.646 [main] WARN o.e.c.p.rest.RestActionBuilderV3 - No fields for object definition: individual
21:58:52.646 [main] WARN o.e.c.p.rest.RestActionBuilderV3 - No fields for object definition: party
21:58:52.647 [main] WARN o.e.c.p.rest.RestActionBuilderV3 - No fields for object definition: event
21:58:52.648 [main] WARN o.e.c.p.rest.RestActionBuilderV3 - No fields for object definition: partyRole
21:58:52.649 [main] WARN o.e.c.p.rest.RestActionBuilderV3 - No fields for object definition: validFor
21:58:52.650 [main] WARN o.e.c.p.rest.RestActionBuilderV3 - No fields for object definition: attachment
21:58:52.652 [main] WARN o.e.c.p.rest.RestActionBuilderV3 - No fields for object definition: status
* There are 12 usable RESTful API endpoints defined in the schema configuration
21:59:14.193 [main] WARN o.e.c.p.rest.RestActionBuilderV3 - No fields for object definition: characteristic
21:59:14.200 [main] WARN o.e.c.p.rest.RestActionBuilderV3 - No fields for object definition: existsDuring
21:59:14.200 [main] WARN o.e.c.p.rest.RestActionBuilderV3 - No fields for object definition: organizationParentRelationship
21:59:14.221 [main] WARN o.e.c.p.rest.RestActionBuilderV3 - No fields for object definition: taxExemptionCertificate
21:59:14.222 [main] WARN o.e.c.p.rest.RestActionBuilderV3 - No fields for object definition: creditRating_item
21:59:14.222 [main] WARN o.e.c.p.rest.RestActionBuilderV3 - No fields for object definition: engagedParty
21:59:14.223 [main] WARN o.e.c.p.rest.RestActionBuilderV3 - No fields for object definition: partyRoleSpecification
21:59:14.224 [main] WARN o.e.c.p.rest.RestActionBuilderV3 - No fields for object definition: creditProfile_item
21:59:14.243 [main] WARN o.e.c.p.rest.RestActionBuilderV3 - No fields for object definition: size
21:59:14.266 [main] WARN o.e.c.p.rest.RestActionBuilderV3 - No fields for object definition: source
21:59:14.267 [main] WARN o.e.c.p.rest.RestActionBuilderV3 - No fields for object definition: reportingSystem
21:59:14.322 [main] WARN o.e.c.p.rest.RestActionBuilderV3 - No fields for object definition: value_item
21:59:14.325 [main] WARN o.e.c.p.rest.RestActionBuilderV3 - No fields for object definition: value
21:59:14.328 [main] WARN o.e.c.p.rest.RestActionBuilderV3 - No fields for object definition: organization
21:59:14.412 [main] WARN o.e.c.p.rest.RestActionBuilderV3 - No fields for object definition: account_item
21:59:14.413 [main] WARN o.e.c.p.rest.RestActionBuilderV3 - No fields for object definition: agreement_item
21:59:14.413 [main] WARN o.e.c.p.rest.RestActionBuilderV3 - No fields for object definition: paymentMethod_item
21:59:14.473 [main] WARN o.e.c.p.rest.RestActionBuilderV3 - No type/format information provided for 'value'. Defaulting to 'string'
21:59:14.480 [main] WARN o.e.c.p.rest.RestActionBuilderV3 - No fields for object definition: BaseEvent_allOf_event
21:59:14.480 [main] WARN o.e.c.p.rest.RestActionBuilderV3 - No fields for object definition: BaseEvent_FVO_allOf_event
21:59:14.481 [main] WARN o.e.c.p.rest.RestActionBuilderV3 - No fields for object definition: Event_allOf_event
21:59:14.481 [main] WARN o.e.c.p.rest.RestActionBuilderV3 - No fields for object definition: Event_FVO_allOf_event
21:59:14.482 [main] WARN o.e.c.p.rest.RestActionBuilderV3 - No fields for object definition: ObjectCharacteristic_allOf_value
21:59:14.482 [main] WARN o.e.c.p.rest.RestActionBuilderV3 - No fields for object definition: ObjectCharacteristic_FVO_allOf_value
* ←[33m[WARNING] ←[0m←[33mUsing experimental settings. Those might not work as expected, or simply straight out crash. Furthermore, they might simply be incomplete features still under development. Used experimental settings: [bbProbabilityUseDataPool, wbProbabilityUseDataPool]←[0m
* Starting to generate test cases
←[1A←[2K* Consumed search budget: 0.037%; covered targets: 0; time per test: 209.0ms (1.0 actions)
←[1A←[2K* Consumed search budget: 99.934%; covered targets: 15; time per test: 2.9ms (1.0 actions)
* Starting to apply minimization phase
* Recomputing full coverage for 15 tests
* No test to minimize
* Minimization phase took 1 seconds
* Evaluated tests: 69371
* Evaluated actions: 70039
* Needed budget: 26%
* Passed time (seconds): 601
* Execution time per test (ms): Avg=3.84 , min=0.00 , max=651.00
* Execution time per action (ms): Avg=3.61 , min=0.00 , max=263.00
* Computation overhead between tests (ms): Avg=4.82 , min=0.00 , max=14935.00
* Computation overhead of resetting the SUT (ms): Avg=0.92 , min=0.00 , max=22.00
* Computation overhead of fetching test results, per test, subset of targets (ms): Avg=0.64 , min=0.00 , max=35.00
* Going to save 16 tests to C:\Users\aruvic\Documents\Development\new_folder\UC001\JAVA\tmf_api_tmf632_party_management_v5.0.0_server\test_output
22:09:15.970 [main] WARN o.e.c.o.service.HttpWsTestCaseWriter - Currently no assertions are generated for response type: text/html;charset=utf-8
* Potential faults: 1
* Covered targets (lines, branches, faults, etc.): 810
* Bytecode line coverage: 3% (346 out of 9961 in 471 units/classes)
* Successfully executed (HTTP code 2xx) 1 endpoints out of 13 (8%)
* EvoMaster process has completed successfully
* Use ←[32m--help←[0m and visit ←[34mhttp://www.evomaster.org←[0m to learn more about available optionsDriver this time threw following error. Can this missing Kafka Client dependency be safely ignored?
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v3.1.3)
2024-05-15T21:57:59.885+02:00 INFO 17572 --- [tp1825738663-22] o.s.boot.SpringApplication : Starting application using Java 17.0.11 with PID 17572 (started by aruvic in C:\Users\aruvic\Documents\Development\new_folder\UC001\JAVA\tmf_api_tmf632_party_management_v5.0.0_server)
2024-05-15T21:57:59.889+02:00 INFO 17572 --- [tp1825738663-22] o.s.boot.SpringApplication : No active profile set, falling back to 1 default profile: "default"
ERROR - Failed to instrument io.micrometer.core.instrument.binder.kafka.KafkaMetrics
java.lang.RuntimeException: Class not found: org/apache/kafka/common/MetricName: java.lang.NullPointerException: Class not found org/apache/kafka/common/MetricName
at org.evomaster.client.java.instrumentation.ComputeClassWriter.getCommonSuperClass(ComputeClassWriter.java:64)
at shaded.org.objectweb.asm.SymbolTable.addMergedType(SymbolTable.java:1202)
at shaded.org.objectweb.asm.Frame.merge(Frame.java:1299)
at shaded.org.objectweb.asm.Frame.merge(Frame.java:1197)
at shaded.org.objectweb.asm.MethodWriter.computeAllFrames(MethodWriter.java:1610)
at shaded.org.objectweb.asm.MethodWriter.visitMaxs(MethodWriter.java:1546)
at shaded.org.objectweb.asm.tree.MethodNode.accept(MethodNode.java:769)
at shaded.org.objectweb.asm.commons.JSRInlinerAdapter.visitEnd(JSRInlinerAdapter.java:158)
at shaded.org.objectweb.asm.MethodVisitor.visitEnd(MethodVisitor.java:783)
at shaded.org.objectweb.asm.MethodVisitor.visitEnd(MethodVisitor.java:783)
at shaded.org.objectweb.asm.tree.MethodNode.accept(MethodNode.java:772)
at shaded.org.objectweb.asm.tree.MethodNode.accept(MethodNode.java:649)
at shaded.org.objectweb.asm.tree.ClassNode.accept(ClassNode.java:452)
at org.evomaster.client.java.instrumentation.Instrumentator.transformBytes(Instrumentator.java:75)
at org.evomaster.client.java.instrumentation.InstrumentingAgent$TransformerForTests.transform(InstrumentingAgent.java:148)
at java.instrument/java.lang.instrument.ClassFileTransformer.transform(ClassFileTransformer.java:244)
at java.instrument/sun.instrument.TransformerManager.transform(TransformerManager.java:188)
at java.instrument/sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:541)
at java.base/java.lang.ClassLoader.defineClass1(Native Method)
at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1017)
at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:150)
at java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:862)
at java.base/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(BuiltinClassLoader.java:760)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:681)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:639)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:525)
at java.base/java.lang.ClassLoader.defineClass1(Native Method)
at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1017)
at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:150)
at java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:862)
at java.base/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(BuiltinClassLoader.java:760)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:681)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:639)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:525)
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:467)
at org.springframework.boot.autoconfigure.condition.FilteringSpringBootCondition.resolve(FilteringSpringBootCondition.java:108)
at org.springframework.boot.autoconfigure.condition.FilteringSpringBootCondition$ClassNameFilter.isPresent(FilteringSpringBootCondition.java:140)
at org.springframework.boot.autoconfigure.condition.FilteringSpringBootCondition$ClassNameFilter$2.matches(FilteringSpringBootCondition.java:128)
at org.springframework.boot.autoconfigure.condition.OnClassCondition$StandardOutcomesResolver.getOutcome(OnClassCondition.java:223)
at org.springframework.boot.autoconfigure.condition.OnClassCondition$StandardOutcomesResolver.getOutcome(OnClassCondition.java:210)
at org.springframework.boot.autoconfigure.condition.OnClassCondition$StandardOutcomesResolver.getOutcomes(OnClassCondition.java:197)
at org.springframework.boot.autoconfigure.condition.OnClassCondition$StandardOutcomesResolver.resolveOutcomes(OnClassCondition.java:186)
at org.springframework.boot.autoconfigure.condition.OnClassCondition$ThreadedOutcomesResolver.lambda$new$0(OnClassCondition.java:146)
at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: java.lang.NullPointerException: Class not found org/apache/kafka/common/MetricName
at org.evomaster.client.java.instrumentation.ComputeClassWriter.typeInfo(ComputeClassWriter.java:200)
2024-05-15T21:58:19.423+02:00 INFO 17572 --- [tp1825738663-22] o.a.c.c.C.[.[localhost].[/aplikacija] : Initializing Spring embedded WebApplicationContext
2024-05-15T21:58:19.424+02:00 INFO 17572 --- [tp1825738663-22] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 16509 ms
g embedded WebApplicationContext
2024-05-15T21:58:19.424+02:00 INFO 17572 --- [tp1825738663-22] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 16509 ms
2024-05-15T21:58:21.150+02:00 INFO 17572 --- [tp1825738663-22] o.hibernate.jpa.internal.util.LogHelper : HHH000204: Processing PersistenceUnitInfo [name: default]
2024-05-15T21:58:21.783+02:00 INFO 17572 --- [tp1825738663-22] org.hibernate.Version : HHH000412: Hibernate ORM core version 6.4.4.Final
2024-05-15T21:58:22.213+02:00 INFO 17572 --- [tp1825738663-22] o.h.c.internal.RegionFactoryInitiator : HHH000026: Second-level cache disabled
2024-05-15T21:58:23.839+02:00 INFO 17572 --- [tp1825738663-22] o.s.o.j.p.SpringPersistenceUnitInfo : No LoadTimeWeaver setup: ignoring JPA class transformer
2024-05-15T21:58:24.053+02:00 INFO 17572 --- [tp1825738663-22] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Starting...
2024-05-15T21:58:25.457+02:00 INFO 17572 --- [tp1825738663-22] com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Added connection com.mysql.cj.jdbc.ConnectionImpl@656f5eda
2024-05-15T21:58:25.461+02:00 INFO 17572 --- [tp1825738663-22] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Start completed.
2024-05-15T21:58:25.865+02:00 WARN 17572 --- [tp1825738663-22] org.hibernate.orm.deprecation : HHH90000025: MySQLDialect does not need to be specified explicitly using 'hibernate.dialect' (remove the property setting and it will be selected by default)
2024-05-15T21:58:35.913+02:00 INFO 17572 --- [tp1825738663-22] o.h.e.t.j.p.i.JtaPlatformInitiator : HHH000489: No JTA platform available (set 'hibernate.transaction.jta.platform' to enable JTA platform integration)
2024-05-15T21:58:37.823+02:00 INFO 17572 --- [tp1825738663-22] j.LocalContainerEntityManagerFactoryBean : Initialized JPA EntityManagerFactory for persistence unit 'default'
2024-05-15T21:58:44.627+02:00 WARN 17572 --- [tp1825738663-22] JpaBaseConfiguration$JpaWebConfiguration : spring.jpa.open-in-view is enabled by default. Therefore, database queries may be performed during view rendering. Explicitly configure spring.jpa.open-in-view to disable this warning
2024-05-15T21:58:48.093+02:00 INFO 17572 --- [tp1825738663-22] o.s.b.a.e.web.EndpointLinksResolver : Exposing 1 endpoint(s) beneath base path '/actuator'
2024-05-15T21:58:48.493+02:00 INFO 17572 --- [tp1825738663-22] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8001 (http) with context path '/aplikacija'
2024-05-15T21:58:48.538+02:00 INFO 17572 --- [tp1825738663-22] o.s.boot.SpringApplication : Started application in 52.588 seconds (process running for 117.1)
2024-05-15T21:58:50.922+02:00 INFO 17572 --- [nio-8001-exec-1] o.a.c.c.C.[.[localhost].[/aplikacija] : Initializing Spring DispatcherServlet 'dispatcherServlet'
2024-05-15T21:58:50.923+02:00 INFO 17572 --- [nio-8001-exec-1] o.s.web.servlet.DispatcherServlet : Initializing Servlet 'dispatcherServlet'
2024-05-15T21:58:50.938+02:00 INFO 17572 --- [nio-8001-exec-1] o.s.web.servlet.DispatcherServlet : Completed initialization in 12 mshi @aruvic ,
regarding your questions:
1) Can I safely ignore the warnings? -> yes
2) Needed budget: 26%, Does this mean I can run this in 26% of 10 minutes and it will achieve same results? -> not exactly (it is a complex story...), but yes, at the high level that is the idea. Can be used as a rough estimate of for how long to run future runs on same API
3) Bytecode line coverage: 3% also expected or this is very low? -> indeed, it is extremely low. in your logs you had a lot of warnings like No fields for object definition: partyOrPartyRole. wondering if those objects are constructed with some OpenAPI syntax we do not handle (which could explain those results). Could it be possible to paste here one of such definitions?
4) Can this missing Kafka Client dependency be safely ignored? -> yes, ignore those. we should actually avoid printing such confusing stack-traces, as it is not an error, but actually something that could be expected to happen in practice