#252: Upgrade extension

kaklakariada commented 11 months ago

Closes #252 Closes #251

morazow commented 11 months ago

It still fails with the following error:

com.exasol.ExaUDFException: F-UDF-CL-SL-JAVA-1080: Exception during run 
alluxio.exception.status.UnavailableException: Failed to connect to master (GrpcServerAddress{HostName=172.18.0.3, SocketAddress=172.18.0.3:19998}) after 44 attempts.Please check if Alluxio master is currently running on "GrpcServerAddress{HostName=172.18.0.3, SocketAddress=172.18.0.3:19998}". Service="FileSystemMasterClient"
alluxio.AbstractClient.connect(AbstractClient.java:321)
alluxio.client.file.BaseFileSystem.rpc(BaseFileSystem.java:635)
alluxio.client.file.BaseFileSystem.createFile(BaseFileSystem.java:174)
alluxio.hadoop.AbstractFileSystem.create(AbstractFileSystem.java:181)
org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1233)
org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1210)
org.apache.parquet.hadoop.util.HadoopOutputFile.create(HadoopOutputFile.java:74)
org.apache.parquet.hadoop.ParquetFileWriter.<init>(ParquetFileWriter.java:347)
org.apache.parquet.hadoop.ParquetFileWriter.<init>(ParquetFileWriter.java:314)
org.apache.parquet.hadoop.ParquetWriter.<init>(ParquetWriter.java:292)
org.apache.parquet.hadoop.ParquetWriter$Builder.build(ParquetWriter.java:680)
com.exasol.cloudetl.parquet.ParquetRowWriter$.apply(ParquetRowWriter.scala:37)
com.exasol.cloudetl.sink.BatchSizedSink$$anon$1.<init>(BatchSizedSink.scala:54)
com.exasol.cloudetl.sink.BatchSizedSink.createWriter(BatchSizedSink.scala:51)
com.exasol.cloudetl.sink.BatchSizedSink.openNewFile(BatchSizedSink.scala:106)
com.exasol.cloudetl.sink.BatchSizedSink.write(BatchSizedSink.scala:75)
com.exasol.cloudetl.scriptclasses.TableDataExporter$.runExport(TableDataExporter.scala:53)
com.exasol.cloudetl.scriptclasses.TableDataExporter$.run(TableDataExporter.scala:36)
com.exasol.cloudetl.scriptclasses.TableDataExporter.run(TableDataExporter.scala)
com.exasol.ExaWrapper.run(ExaWrapper.java:197)
Caused by: alluxio.exception.status.UnavailableException: Failed to handshake with master 172.18.0.3:19998 to load cluster default configuration values: UNKNOWN
alluxio.conf.Configuration.loadConfiguration(Configuration.java:458)
alluxio.ClientContext.loadConf(ClientContext.java:123)
alluxio.ClientContext.loadConfIfNotLoaded(ClientContext.java:147)
alluxio.AbstractClient.beforeConnect(AbstractClient.java:201)
alluxio.AbstractMasterClient.beforeConnect(AbstractMasterClient.java:94)
alluxio.AbstractClient.connect(AbstractClient.java:265)
    ... 19 more
Caused by: io.grpc.StatusRuntimeException: UNKNOWN
io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:271)
io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:252)
io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:165)
alluxio.grpc.MetaMasterConfigurationServiceGrpc$MetaMasterConfigurationServiceBlockingStub.getConfiguration(MetaMasterConfigurationServiceGrpc.java:405)
alluxio.conf.Configuration.loadConfiguration(Configuration.java:452)
    ... 24 more
Caused by: java.lang.NoClassDefFoundError: Could not initialize class io.netty.channel.unix.Errors
io.netty.channel.unix.Socket.connect(Socket.java:351)
io.netty.channel.epoll.AbstractEpollChannel.doConnect0(AbstractEpollChannel.java:773)
io.netty.channel.epoll.EpollSocketChannel.doConnect0(EpollSocketChannel.java:144)
io.netty.channel.epoll.AbstractEpollChannel.doConnect(AbstractEpollChannel.java:758)
io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.connect(AbstractEpollChannel.java:600)
io.netty.channel.DefaultChannelPipeline$HeadContext.connect(DefaultChannelPipeline.java:1342)
io.netty.channel.AbstractChannelHandlerContext.invokeConnect(AbstractChannelHandlerContext.java:653)
io.netty.channel.AbstractChannelHandlerContext.connect(AbstractChannelHandlerContext.java:632)
io.netty.channel.ChannelDuplexHandler.connect(ChannelDuplexHandler.java:54)
io.grpc.netty.WriteBufferingAndExceptionHandler.connect(WriteBufferingAndExceptionHandler.java:157)
io.netty.channel.AbstractChannelHandlerContext.invokeConnect(AbstractChannelHandlerContext.java:655)
io.netty.channel.AbstractChannelHandlerContext.access$1000(AbstractChannelHandlerContext.java:61)
io.netty.channel.AbstractChannelHandlerContext$9.run(AbstractChannelHandlerContext.java:637)
io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:174)
io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:167)
io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470)
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:406)
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
java.base/java.lang.Thread.run(Thread.java:829)
 (Session: 1773761757961715712)
  at com.exasol.jdbc.ExceptionFactory.createSQLException(ExceptionFactory.java:33)
  at com.exasol.jdbc.EXASQLException.getSQLExceptionIntern(EXASQLException.java:50)
  at com.exasol.jdbc.AbstractEXAStatement.execute(AbstractEXAStatement.java:478)
  at com.exasol.jdbc.EXAStatement.execute(EXAStatement.java:289)
  at com.exasol.cloudetl.BaseIntegrationTest.executeStmt(BaseIntegrationTest.scala:52)
  at com.exasol.cloudetl.BaseIntegrationTest.executeStmt$(BaseIntegrationTest.scala:51)
  at com.exasol.cloudetl.alluxio.AlluxioExportImportIT.executeStmt(AlluxioExportImportIT.scala:13)
  at com.exasol.cloudetl.alluxio.AlluxioExportImportIT.exportIntoAlluxio(AlluxioExportImportIT.scala:110)
  at com.exasol.cloudetl.alluxio.AlluxioExportImportIT.$anonfun$new$1(AlluxioExportImportIT.scala:64)
  at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)

I think one of the version updates messed up the classes.

morazow commented 11 months ago

The dependency extension-manager-integration-test-java:0.5.0 seems to be not released to maven yet.

morazow commented 11 months ago

For now, let's not update:

        <dependency>
            <groupId>io.grpc</groupId>
            <artifactId>grpc-netty</artifactId>
            <version>1.56.1</version>
        </dependency>
        <dependency>
            <groupId>org.alluxio</groupId>
            <artifactId>alluxio-core-client-hdfs</artifactId>
            <version>300</version>
        </dependency>

They are breaking for aluxio docker integration tests.

Created #263 to separately address them.