search

Alluxio / alluxio

Alluxio, data orchestration for analytics and machine learning in the cloud
https://www.alluxio.io
Apache License 2.0
6.81k stars 2.94k forks source link

alluxio runTests failed about alluxio@301 on centos8_aarch64 #17818

Open Tom-python0121 opened 1 year ago

Tom-python0121 commented 1 year ago

Hello, I meet a problem:alluxio runTests failed about alluxio@301 on centos8_aarch64

Steps to reproduce the issue

comand:
1.wget https://downloads.alluxio.io/downloads/files/301/alluxio-301-bin.tar.gz
cd alluxio-301
yum install openjdk@11.0.11
2.hostname
3.hostname localhost
alluxio_home = cls.spack_operator.get_bin_home(app_hash=cls.app_hash)
java_home = cls.spack_operator.get_bin_home(app_hash=cls.java_hash)
4.rm -f {alluxio_home}/conf/alluxio-site.properties
5.cp {alluxio_home}/conf/alluxio-site.properties.template {alluxio_home}/conf/alluxio-site.properties
6.echo "alluxio.master.hostname=localhost" >>  {alluxio_home}/conf/alluxio-site.properties
7.rm -f {alluxio_home}/conf/alluxio-env.sh
8.cp {alluxio_home}/conf/alluxio-env.sh.template {alluxio_home}/conf/alluxio-env.sh
9.echo "export JAVA_HOME={java_home}" >>  {alluxio_home}/conf/alluxio-env.sh
10.echo "export JAVA_HOME={java_home}" >>  {alluxio_home}/conf/alluxio-site.properties
11.echo "export ALLUXIO_MASTER_ADDRESS=localhost" >> {alluxio_home}/conf/alluxio-site.properties
12.echo "export ALLUXIO_WORKER_MEMORY_SIZE=10240M" >> {alluxio_home}/conf/alluxio-site.properties
13.配免密等操作
if os.path.exists('/root/.ssh/id_rsa.pub'):
    cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
    chmod 0600 ~/.ssh/authorized_keys
    echo  "StrictHostKeyChecking=no" >> ~/.ssh/config
else:
    ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
    cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
    chmod 0600 ~/.ssh/authorized_keys
    sed -i "/StrictHostKeyChecking=no/d" ~/.ssh/config
    echo  "StrictHostKeyChecking=no" >> ~/.ssh/config
14.alluxio-mount.sh Mount
15.ulimit -n 65000
16.alluxio format
17.alluxio-start.sh local && sleep 10
18.alluxio runTests

error:
Failed to load jar /home/spack/opt/spack/linux-centos8-aarch64/gcc-8.5.0/alluxio-301-3lcwirpzhnm2wve3rircpg6tlfdt5q75/lib/alluxio-underfs-hadoop-cosn-3.1.0-5.8.5-301.jar: java.lang.NoClassDefFoundError: alluxio/underfs/hdfs/HdfsUnderFileSystemFactory
2023-07-21 16:07:35,700 ERROR [main](RunTestUtils.java:64) - Exception running test: alluxio.cli.BasicOperations@333cb916
java.nio.BufferUnderflowException
        at java.base/java.nio.Buffer.nextGetIndex(Buffer.java:651)
        at java.base/java.nio.HeapByteBuffer.getInt(HeapByteBuffer.java:402)
        at alluxio.cli.BasicOperations.readFile(BasicOperations.java:113)
        at alluxio.cli.BasicOperations.call(BasicOperations.java:85)
        at alluxio.cli.BasicOperations.call(BasicOperations.java:43)
        at alluxio.cli.RunTestUtils.runExample(RunTestUtils.java:62)
        at alluxio.cli.TestRunner.runTest(TestRunner.java:234)
        at alluxio.cli.TestRunner.runTests(TestRunner.java:153)
        at alluxio.cli.TestRunner.main(TestRunner.java:115)
Failed the test!
runTest --operation BASIC_NON_BYTE_BUFFER --readType CACHE_PROMOTE --writeType MUST_CACHE
2023-07-21 16:07:35,765 INFO  [main](BasicNonByteBufferOperations.java:100) - writeFile to file /default_tests_files/BASIC_NON_BYTE_BUFFER_CACHE_PROMOTE_MUST_CACHE took 48 ms.
2023-07-21 16:07:35,822 ERROR [main](RunTestUtils.java:64) - Exception running test: alluxio.cli.BasicNonByteBufferOperations@2764c546
java.io.EOFException
        at java.base/java.io.DataInputStream.readInt(DataInputStream.java:397)
        at alluxio.cli.BasicNonByteBufferOperations.read(BasicNonByteBufferOperations.java:137)
        at alluxio.cli.BasicNonByteBufferOperations.call(BasicNonByteBufferOperations.java:88)
        at alluxio.cli.BasicNonByteBufferOperations.call(BasicNonByteBufferOperations.java:51)
        at alluxio.cli.RunTestUtils.runExample(RunTestUtils.java:62)
        at alluxio.cli.TestRunner.runTest(TestRunner.java:238)
        at alluxio.cli.TestRunner.runTests(TestRunner.java:153)
        at alluxio.cli.TestRunner.main(TestRunner.java:115)
Failed the test!
runTest --operation BASIC --readType CACHE_PROMOTE --writeType CACHE_THROUGH
2023-07-21 16:07:35,997 INFO  [main](BasicOperations.java:100) - writeFile to file /default_tests_files/BASIC_CACHE_PROMOTE_ASYNC_THROUGH took 11 ms.
2023-07-21 16:07:36,002 ERROR [main](RunTestUtils.java:64) - Exception running test: alluxio.cli.BasicOperations@557286ad
java.nio.BufferUnderflowException
        at java.base/java.nio.Buffer.nextGetIndex(Buffer.java:651)
        at java.base/java.nio.HeapByteBuffer.getInt(HeapByteBuffer.java:402)
        at alluxio.cli.BasicOperations.readFile(BasicOperations.java:113)
        at alluxio.cli.BasicOperations.call(BasicOperations.java:85)
        at alluxio.cli.BasicOperations.call(BasicOperations.java:43)
        at alluxio.cli.RunTestUtils.runExample(RunTestUtils.java:62)
        at alluxio.cli.TestRunner.runTest(TestRunner.java:234)
        at alluxio.cli.TestRunner.runTests(TestRunner.java:153)
        at alluxio.cli.TestRunner.main(TestRunner.java:115)
Failed the test!
runTest --operation BASIC_NON_BYTE_BUFFER --readType CACHE_PROMOTE --writeType ASYNC_THROUGH
2023-07-21 16:07:36,050 INFO  [main](BasicNonByteBufferOperations.java:100) - writeFile to file /default_tests_files/BASIC_NON_BYTE_BUFFER_CACHE_PROMOTE_ASYNC_THROUGH took 38 ms.
2023-07-21 16:07:36,061 ERROR [main](RunTestUtils.java:64) - Exception running test: alluxio.cli.BasicNonByteBufferOperations@4682eba5
java.io.EOFException
        at java.base/java.io.DataInputStream.readInt(DataInputStream.java:397)
        at alluxio.cli.BasicNonByteBufferOperations.read(BasicNonByteBufferOperations.java:137)
        at alluxio.cli.BasicNonByteBufferOperations.call(BasicNonByteBufferOperations.java:88)
        at alluxio.cli.BasicNonByteBufferOperations.call(BasicNonByteBufferOperations.java:51)
        at alluxio.cli.RunTestUtils.runExample(RunTestUtils.java:62)
        at alluxio.cli.TestRunner.runTest(TestRunner.java:238)
        at alluxio.cli.TestRunner.runTests(TestRunner.java:153)
        at alluxio.cli.TestRunner.main(TestRunner.java:115)
Failed the test!
runTest --operation BASIC --readType CACHE --writeType MUST_CACHE
2023-07-21 16:07:36,073 INFO  [main](BasicOperations.java:100) - writeFile to file /default_tests_files/BASIC_CACHE_MUST_CACHE took 11 ms.
2023-07-21 16:07:36,076 ERROR [main](RunTestUtils.java:64) - Exception running test: alluxio.cli.BasicOperations@2125ad3
java.nio.BufferUnderflowException
        at java.base/java.nio.Buffer.nextGetIndex(Buffer.java:651)
        at java.base/java.nio.HeapByteBuffer.getInt(HeapByteBuffer.java:402)
        at alluxio.cli.BasicOperations.readFile(BasicOperations.java:113)
        at alluxio.cli.BasicOperations.call(BasicOperations.java:85)
        at alluxio.cli.BasicOperations.call(BasicOperations.java:43)
        at alluxio.cli.RunTestUtils.runExample(RunTestUtils.java:62)
        at alluxio.cli.TestRunner.runTest(TestRunner.java:234)
        at alluxio.cli.TestRunner.runTests(TestRunner.java:153)
        at alluxio.cli.TestRunner.main(TestRunner.java:115)
Failed the test!
runTest --operation BASIC_NON_BYTE_BUFFER --readType CACHE --writeType MUST_CACHE
2023-07-21 16:07:36,117 INFO  [main](BasicNonByteBufferOperations.java:100) - writeFile to file /default_tests_files/BASIC_NON_BYTE_BUFFER_CACHE_MUST_CACHE took 30 ms.
2023-07-21 16:07:36,125 ERROR [main](RunTestUtils.java:64) - Exception running test: alluxio.cli.BasicNonByteBufferOperations@2f2d52ef
java.io.EOFException
        at java.base/java.io.DataInputStream.readInt(DataInputStream.java:397)
        at alluxio.cli.BasicNonByteBufferOperations.read(BasicNonByteBufferOperations.java:137)
        at alluxio.cli.BasicNonByteBufferOperations.call(BasicNonByteBufferOperations.java:88)
        at alluxio.cli.BasicNonByteBufferOperations.call(BasicNonByteBufferOperations.java:51)
        at alluxio.cli.RunTestUtils.runExample(RunTestUtils.java:62)
        at alluxio.cli.TestRunner.runTest(TestRunner.java:238)
        at alluxio.cli.TestRunner.runTests(TestRunner.java:153)
        at alluxio.cli.TestRunner.main(TestRunner.java:115)
Failed the test!

The 300 version test is OK, but the 301 version test fails. Can you tell me how to fix it?

jja725 commented 1 year ago

Hi, thanks for raising the issue, do you what happened for runTest for the newer version?@huanghua78

huanghua78 commented 1 year ago

Thank you @Tom-python0121 for report this issue. We will investigate and fix it. I have verified this is a valid bug I will ask @JiamingMai to fix it.

fubingting commented 1 year ago

Excuse me, is there any concern about solving this problem? The 300 version passes the test, but the 301, 302, and 303 versions report errors.@huanghua78