Audit log failing with code changes due to Spark 3

[miroslavpojer]

Issue observed in branch: feature/2037-develop-spark-3-redone. After merge of this issue there were agreement to re-test develop for this issue again.

Preconditions:

• Menas UI is running
• Schemas menu visible
• User can click to New Schema

Steps:

• User click to New Schema button and fill form.
• User click select newly created schema in menu and click to Edit button.
• Edit update description field and confirm.
• User will wait or user click to Audit trail button

Expected:

• when waiting no failure visible
• when clicked to Audit trail then full history is visible

Actual:

• when waiting - error dialog shown in short time
• when clicked to Audit trail then User can see loading Audit trail table and in short time error dialog

Tips from later (API) testing (black code approach)

• similar problem found after disabling a schema ==> problem with creation of schema versions
• API test details: Request timeout expired.
• observed on v2 and v3 API
• Same problem found for Dataset, Mapping Rule and Dataset Properties (where get trail version is not available in Menas UI)

Observed error dialog:

Server console output: java.lang.NoSuchMethodError: scala.Predef$.refArrayOps([Ljava/lang/Object;)Lscala/collection/mutable/ArrayOps; at za.co.absa.enceladus.model.backend.audit.Auditable.getFieldIndex(Auditable.scala:36) at za.co.absa.enceladus.model.backend.audit.Auditable.$anonfun$getPrimitiveFieldsAudit$1(Auditable.scala:46) at scala.collection.immutable.List.map(List.scala:293) at za.co.absa.enceladus.model.backend.audit.Auditable.getPrimitiveFieldsAudit(Auditable.scala:45) at za.co.absa.enceladus.model.backend.audit.Auditable.getPrimitiveFieldsAudit$(Auditable.scala:44) at za.co.absa.enceladus.model.Schema.getAuditMessages(Schema.scala:67) at za.co.absa.enceladus.model.Schema.getAuditMessages(Schema.scala:25) at za.co.absa.enceladus.rest_api.services.VersionedModelService.$anonfun$getAuditTrail$2(VersionedModelService.scala:180) at scala.collection.Iterator$$anon$10.next(Iterator.scala:461) at scala.collection.Iterator.toStream(Iterator.scala:1417) at scala.collection.Iterator.toStream$(Iterator.scala:1416) at scala.collection.AbstractIterator.toStream(Iterator.scala:1431) at scala.collection.TraversableOnce.toSeq(TraversableOnce.scala:336) at scala.collection.TraversableOnce.toSeq$(TraversableOnce.scala:336) at scala.collection.AbstractIterator.toSeq(Iterator.scala:1431) at za.co.absa.enceladus.rest_api.services.VersionedModelService.$anonfun$getAuditTrail$1(VersionedModelService.scala:180) at scala.concurrent.Future.$anonfun$flatMap$1(Future.scala:307) at scala.concurrent.impl.Promise.$anonfun$transformWith$1(Promise.scala:41) at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64) at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402) at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056) at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692) at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) [ERROR] 2022-12-13 14:56:03,975 [http-nio-8080-exec-8] za.co.absa.enceladus.rest_api.controllers.RestExceptionHandler - Exception: RestResponse(Request timeout expired.,Some(RequestTimeoutExpiredError(request_timeout_expired)),2022-12-13T13:56:03.975Z[UTC],dce3eee5-3fd9-41da-af4c-5016d0d1274e) org.springframework.web.context.request.async.AsyncRequestTimeoutException at org.springframework.web.context.request.async.TimeoutDeferredResultProcessingInterceptor.handleTimeout(TimeoutDeferredResultProcessingInterceptor.java:42) at org.springframework.web.context.request.async.DeferredResultInterceptorChain.triggerAfterTimeout(DeferredResultInterceptorChain.java:79) at org.springframework.web.context.request.async.WebAsyncManager.lambda$startDeferredResultProcessing$5(WebAsyncManager.java:390) at java.util.ArrayList.forEach(ArrayList.java:1259) at org.springframework.web.context.request.async.StandardServletAsyncWebRequest.onTimeout(StandardServletAsyncWebRequest.java:151) at org.apache.catalina.core.AsyncListenerWrapper.fireOnTimeout(AsyncListenerWrapper.java:44) at org.apache.catalina.core.AsyncContextImpl.timeout(AsyncContextImpl.java:135) at org.apache.catalina.connector.CoyoteAdapter.asyncDispatch(CoyoteAdapter.java:147) at org.apache.coyote.AbstractProcessor.dispatch(AbstractProcessor.java:241) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:59) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:890) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1789) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191) at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:750)

[jakipatryk]

Problem was probably caused by wrong Scala version on the local machine. Closing as I couldn't reproduce it and neither @miroslavpojer could after Scala version change.