search

GenomicsDB / GenomicsDB

High performance data storage for importing, querying and transforming variants.
https://genomicsdb.readthedocs.io
Other
94 stars 18 forks source link

Support MacOS/Linux arm64 architecture #204

Open nalinigans opened 2 years ago

nalinigans commented 2 years ago

gatk forum users have started requesting support for MacOS and Linux arm64 architectures - see forum post.

lima1 commented 1 year ago

Probably not super helpful me posting this since it's not supported yet, but compilation on arm64 fails for me here (both master and develop).

#0 4052.2 [ 61%] Creating directories for 'crc32-build'
#0 4052.2 [ 61%] Performing download step (download, verify and extract) for 'crc32-build'
#0 4052.2 -- Downloading...
#0 4052.2    dst='/root/gcssdk-install/v1.24.0/src/1.1.0.tar.gz'
#0 4052.2    timeout='none'
#0 4052.2    inactivity timeout='none'
#0 4052.2 -- Using src='https://github.com/google/crc32c/archive/1.1.0.tar.gz'
#0 4052.9 -- Downloading... done
#0 4053.0 -- extracting...
#0 4053.0      src='/root/gcssdk-install/v1.24.0/src/1.1.0.tar.gz'
#0 4053.0      dst='/root/gcssdk-install/v1.24.0/src/crc32-build'
#0 4053.0 -- extracting... [tar xfz]
#0 4053.0 -- extracting... [analysis]
#0 4053.0 -- extracting... [rename]
#0 4053.0 -- extracting... [clean up]
#0 4053.0 -- extracting... done
#0 4053.0 [ 61%] No update step for 'crc32-build'
#0 4053.0 [ 61%] No patch step for 'crc32-build'
#0 4053.1 [ 61%] Performing configure step for 'crc32-build'
#0 4053.1 -- The C compiler identification is GNU 11.3.0
#0 4053.2 -- The CXX compiler identification is GNU 11.3.0
#0 4053.2 -- Detecting C compiler ABI info
#0 4053.2 -- Detecting C compiler ABI info - done
#0 4053.2 -- Check for working C compiler: /usr/bin/cc - skipped
#0 4053.2 -- Detecting C compile features
#0 4053.2 -- Detecting C compile features - done
#0 4053.2 -- Detecting CXX compiler ABI info
#0 4053.3 -- Detecting CXX compiler ABI info - done
#0 4053.3 -- Check for working CXX compiler: /usr/bin/c++ - skipped
#0 4053.3 -- Detecting CXX compile features
#0 4053.3 -- Detecting CXX compile features - done
#0 4053.3 -- Performing Test CRC32C_HAVE_NO_DEPRECATED
#0 4053.3 -- Performing Test CRC32C_HAVE_NO_DEPRECATED - Success
#0 4053.3 -- Performing Test CRC32C_HAVE_NO_SIGN_COMPARE
#0 4053.4 -- Performing Test CRC32C_HAVE_NO_SIGN_COMPARE - Success
#0 4053.4 -- Performing Test CRC32C_HAVE_NO_UNUSED_PARAMETER
#0 4053.4 -- Performing Test CRC32C_HAVE_NO_UNUSED_PARAMETER - Success
#0 4053.4 -- Performing Test CRC32C_HAVE_NO_MISSING_FIELD_INITIALIZERS
#0 4053.5 -- Performing Test CRC32C_HAVE_NO_MISSING_FIELD_INITIALIZERS - Success
#0 4053.5 -- Performing Test HAVE_BUILTIN_PREFETCH
#0 4053.5 -- Performing Test HAVE_BUILTIN_PREFETCH - Success
#0 4053.5 -- Performing Test HAVE_MM_PREFETCH
#0 4053.6 -- Performing Test HAVE_MM_PREFETCH - Failed
#0 4053.6 -- Performing Test HAVE_SSE42
#0 4053.6 -- Performing Test HAVE_SSE42 - Failed
#0 4053.6 -- Performing Test HAVE_ARM64_CRC32C
#0 4053.8 -- Performing Test HAVE_ARM64_CRC32C - Success
#0 4053.8 -- Performing Test HAVE_STRONG_GETAUXVAL
#0 4053.9 -- Performing Test HAVE_STRONG_GETAUXVAL - Success
#0 4053.9 -- Performing Test HAVE_WEAK_GETAUXVAL
#0 4054.0 -- Performing Test HAVE_WEAK_GETAUXVAL - Success
#0 4054.0 -- Configuring done
#0 4054.0 -- Generating done
#0 4054.0 -- Build files have been written to: /root/gcssdk-install/v1.24.0/src/crc32-build-build
#0 4054.0 [ 61%] Performing build step for 'crc32-build'
#0 4054.0 [ 20%] Building CXX object CMakeFiles/crc32c_arm64.dir/src/crc32c_arm64.cc.o
#0 4054.2 [ 20%] Built target crc32c_arm64
#0 4054.2 [ 40%] Building CXX object CMakeFiles/crc32c_sse42.dir/src/crc32c_sse42.cc.o
#0 4054.2 [ 40%] Built target crc32c_sse42
#0 4054.2 [ 60%] Building CXX object CMakeFiles/crc32c.dir/src/crc32c_portable.cc.o
#0 4054.3 [ 80%] Building CXX object CMakeFiles/crc32c.dir/src/crc32c.cc.o
#0 4054.3 In file included from /root/gcssdk-install/v1.24.0/src/crc32-build/src/crc32c.cc:11:
#0 4054.3 /root/gcssdk-install/v1.24.0/src/crc32-build/src/./crc32c_arm64_linux_check.h: In function ‘bool crc32c::CanUseArm64Linux()’:
#0 4054.3 /root/gcssdk-install/v1.24.0/src/crc32-build/src/./crc32c_arm64_linux_check.h:36:37: error: the address of ‘long unsigned int getauxval(long unsigned int)’ will never be NULL [-Werror=address]
#0 4054.3    36 |   unsigned long hwcap = (&getauxval != nullptr) ? getauxval(AT_HWCAP) : 0;
#0 4054.3       |                          ~~~~~~~~~~~^~~~~~~~~~
#0 4054.4 cc1plus: all warnings being treated as errors
#0 4054.4 make[5]: *** [CMakeFiles/crc32c.dir/build.make:90: CMakeFiles/crc32c.dir/src/crc32c.cc.o] Error 1
#0 4054.4 make[4]: *** [CMakeFiles/Makefile2:140: CMakeFiles/crc32c.dir/all] Error 2
#0 4054.4 make[3]: *** [Makefile:136: all] Error 2
#0 4054.4 make[2]: *** [dependencies/TileDB/CMakeFiles/crc32-build.dir/build.make:86: /root/gcssdk-install/v1.24.0/src/crc32-build-stamp/crc32-build-build] Error 2
#0 4054.4 make[1]: *** [CMakeFiles/Makefile2:822: dependencies/TileDB/CMakeFiles/crc32-build.dir/all] Error 2
#0 4054.4 make: *** [Makefile:146: all] Error 2
------
nalinigans commented 1 year ago

@lima1, just last week built GenomicsDB on a macOS m2 arm64 and it built fine using clang although I had a couple of tests failing. Can you please add the following information? Thanks.

lima1 commented 1 year ago

Just tried my amd64 Dockerimage on my new M2 MacBook Pro, although on a Linux image.

Here's how I tried to compile:

ENV GENOMICSDB_PATH=/opt/GenomicsDB
ENV GENOMICSDB_BRANCH=develop
RUN mkdir $GENOMICSDB_PATH
ENV INSTALL_PREFIX=$GENOMICSDB_PATH
ENV PREREQS_ENV=$GENOMICSDB_PATH/genomicsdb_prereqs.sh
ENV JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64
ENV JAVA_HOME=/usr/lib/jvm/java-17-openjdk-arm64
ENV MAVEN_VERSION=3.9.2

RUN ls $JAVA_HOME

WORKDIR /tmp

RUN git clone --recursive --branch $GENOMICSDB_BRANCH https://github.com/GenomicsDB/GenomicsDB.git && \
    cd GenomicsDB/scripts/prereqs && \
    ./install_prereqs.sh && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

RUN chmod +x $PREREQS_ENV && \
    $PREREQS_ENV && \
    cmake -DCMAKE_INSTALL_PREFIX=$GENOMICSDB_PATH -DCMAKE_BUILD_TYPE=Release ./GenomicsDB && \
    make && make install && \
    rm -rf /tmp/GenomicsDB
nalinigans commented 1 year ago

@lima1, can you please try the develop branch and see if that fixes your issues building on amd64 Dockerimage on my new M2 MacBook Pro? Thanks.

lima1 commented 1 year ago

Thanks @nalinigans. It compiled successfully last week, including the R bindings. Haven’t thoroughly tested yet. Any recent changes since then that I should try?

nalinigans commented 1 year ago

Thanks @lima1. Nothing recently added yet. But, you don't need to specify reference genome unless you are generating a vcf. I will be adding support for a protobuf export configuration later, this may be useful as you can use genomic coordinates directly, in addition to setting segment sizes, etc. almost like what GenotypeGVCFs offers.

mr-c commented 1 year ago

Hello from Debian. We are able to compile genomicsdb 1.5.0 for arm64/aarch64 but we get test failures:

Here are some excepts from the build log ``` test 5 Start 5: test_array_iterator 5: Test command: /<>/obj-aarch64-linux-gnu/genomicsdb-tiledb/test/test_array_iterator 5: Working Directory: /<>/obj-aarch64-linux-gnu/genomicsdb-tiledb/test 5: Test timeout computed to be: 10000000 5: [TileDB::utils] Error: (gzip_handle_error) Cannot compress with GZIP: deflateInit error: Z_STREAM_ERROR 5: [TileDB::Codec] Error: Could not compress with GZIP. 5: [TileDB::WriteState] Error: Cannot compress tile for /tmp/TileDBTestzmgk1D/array_iterator_test_ws/test_sparse_array_it_full/.__e38c070f-ad54-40f6-8b1c-b4bdc2e0ef58281473382141984_1688334784891/ATTR_INT32.tdb. 5: 5: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 5: test_array_iterator is a Catch v2.13.10 host application. 5: Run with -? for options 5: 5: ------------------------------------------------------------------------------- 5: Test sparse array iterator 5: ------------------------------------------------------------------------------- 5: ./genomicsdb-tiledb/test/src/c_api/test_array_iterator.cc:193 5: ............................................................................... 5: 5: ./genomicsdb-tiledb/test/src/c_api/test_array_iterator.cc:106: FAILED: 5: CHECK( tiledb_array_write(tiledb_array, buffers, buffer_sizes) == 0 ) 5: with expansion: 5: -1 == 0 5: 5: [TileDB::utils] Error: (gzip_handle_error) Cannot compress with GZIP: deflateInit error: Z_STREAM_ERROR 5: [TileDB::Codec] Error: Could not compress with GZIP. 5: [TileDB::WriteState] Error: Cannot compress tile for /tmp/TileDBTestzmgk1D/array_iterator_test_ws/test_sparse_array_it_full/.__e38c070f-ad54-40f6-8b1c-b4bdc2e0ef58281473382141984_1688334784891/ATTR_INT32.tdb. 5: ./genomicsdb-tiledb/test/src/c_api/test_array_iterator.cc:107: FAILED: 5: CHECK( tiledb_array_finalize(tiledb_array) == 0 ) 5: with expansion: 5: -1 == 0 5: 5: ./genomicsdb-tiledb/test/src/c_api/test_array_iterator.cc:184: FAILED: 5: CHECK( i == nval ) 5: with expansion: 5: 0 == 8 5: 5: [TileDB::utils] Error: (gzip_handle_error) Cannot compress with GZIP: deflateInit error: Z_STREAM_ERROR 5: [TileDB::Codec] Error: Could not compress with GZIP. 5: [TileDB::WriteState] Error: Cannot compress tile for /tmp/TileDBTestNm9PcM/array_iterator_test_ws/test_sparse_array_it_reset/.__f5d7dda5-66cc-45e0-a0b5-aaf415f7c2f6281473382141984_1688334784892/ATTR_INT32.tdb. 5: ------------------------------------------------------------------------------- 5: Test sparse array iterator with reset 5: ------------------------------------------------------------------------------- 5: ./genomicsdb-tiledb/test/src/c_api/test_array_iterator.cc:200 5: ............................................................................... 5: 5: ./genomicsdb-tiledb/test/src/c_api/test_array_iterator.cc:106: FAILED: 5: CHECK( tiledb_array_write(tiledb_array, buffers, buffer_sizes) == 0 ) 5: with expansion: 5: -1 == 0 5: 5: [TileDB::utils] Error: (gzip_handle_error) Cannot compress with GZIP: deflateInit error: Z_STREAM_ERROR 5: [TileDB::Codec] Error: Could not compress with GZIP. 5: [TileDB::WriteState] Error: Cannot compress tile for /tmp/TileDBTestNm9PcM/array_iterator_test_ws/test_sparse_array_it_reset/.__f5d7dda5-66cc-45e0-a0b5-aaf415f7c2f6281473382141984_1688334784892/ATTR_INT32.tdb. 5: ./genomicsdb-tiledb/test/src/c_api/test_array_iterator.cc:107: FAILED: 5: CHECK( tiledb_array_finalize(tiledb_array) == 0 ) 5: with expansion: 5: -1 == 0 5: 5: ./genomicsdb-tiledb/test/src/c_api/test_array_iterator.cc:184: FAILED: 5: CHECK( i == nval ) 5: with expansion: 5: 0 == 8 5: 5: ./genomicsdb-tiledb/test/src/c_api/test_array_iterator.cc:184: FAILED: 5: CHECK( i == nval ) 5: with expansion: 5: 0 == 8 5: 5: [TileDB::utils] Error: (gzip_handle_error) Cannot compress with GZIP: deflateInit error: Z_STREAM_ERROR 5: [TileDB::Codec] Error: Could not compress with GZIP. 5: [TileDB::WriteState] Error: Cannot compress tile for /tmp/TileDBTestHjv52m/array_iterator_test_ws/test_sparse_array_it_reset_subarray/.__110d6e71-2706-4e63-b93b-a5147687fde3281473382141984_1688334784893/ATTR_INT32.tdb. 5: ------------------------------------------------------------------------------- 5: Test sparse array iterator with reset subarray 5: ------------------------------------------------------------------------------- 5: ./genomicsdb-tiledb/test/src/c_api/test_array_iterator.cc:210 5: ............................................................................... 5: 5: ./genomicsdb-tiledb/test/src/c_api/test_array_iterator.cc:106: FAILED: 5: CHECK( tiledb_array_write(tiledb_array, buffers, buffer_sizes) == 0 ) 5: with expansion: 5: -1 == 0 5: 5: [TileDB::utils] Error: (gzip_handle_error) Cannot compress with GZIP: deflateInit error: Z_STREAM_ERROR 5: [TileDB::Codec] Error: Could not compress with GZIP. 5: [TileDB::WriteState] Error: Cannot compress tile for /tmp/TileDBTestHjv52m/array_iterator_test_ws/test_sparse_array_it_reset_subarray/.__110d6e71-2706-4e63-b93b-a5147687fde3281473382141984_1688334784893/ATTR_INT32.tdb. 5: ./genomicsdb-tiledb/test/src/c_api/test_array_iterator.cc:107: FAILED: 5: CHECK( tiledb_array_finalize(tiledb_array) == 0 ) 5: with expansion: 5: -1 == 0 5: 5: ./genomicsdb-tiledb/test/src/c_api/test_array_iterator.cc:184: FAILED: 5: CHECK( i == nval ) 5: with expansion: 5: 0 == 8 5: 5: ./genomicsdb-tiledb/test/src/c_api/test_array_iterator.cc:184: FAILED: 5: CHECK( i == nval ) 5: with expansion: 5: 0 == 5 5: 5: [TileDB::utils] Error: (gzip_handle_error) Cannot compress with GZIP: deflateInit error: Z_STREAM_ERROR 5: [TileDB::Codec] Error: Could not compress with GZIP. 5: [TileDB::WriteState] Error: Cannot compress tile for /tmp/TileDBTestGR6Kaq/array_iterator_test_ws/test_sparse_array_it_filter/.__804d85e2-ab0b-4ae4-8f2d-ab1b6add4505281473382141984_1688334784894/ATTR_INT32.tdb. 5: ------------------------------------------------------------------------------- 5: Test sparse array iterator with filter 5: ------------------------------------------------------------------------------- 5: ./genomicsdb-tiledb/test/src/c_api/test_array_iterator.cc:220 5: ............................................................................... 5: 5: ./genomicsdb-tiledb/test/src/c_api/test_array_iterator.cc:106: FAILED: 5: CHECK( tiledb_array_write(tiledb_array, buffers, buffer_sizes) == 0 ) 5: with expansion: 5: -1 == 0 5: 5: [TileDB::utils] Error: (gzip_handle_error) Cannot compress with GZIP: deflateInit error: Z_STREAM_ERROR 5: [TileDB::Codec] Error: Could not compress with GZIP. 5: [TileDB::WriteState] Error: Cannot compress tile for /tmp/TileDBTestGR6Kaq/array_iterator_test_ws/test_sparse_array_it_filter/.__804d85e2-ab0b-4ae4-8f2d-ab1b6add4505281473382141984_1688334784894/ATTR_INT32.tdb. 5: ./genomicsdb-tiledb/test/src/c_api/test_array_iterator.cc:107: FAILED: 5: CHECK( tiledb_array_finalize(tiledb_array) == 0 ) 5: with expansion: 5: -1 == 0 5: 5: ./genomicsdb-tiledb/test/src/c_api/test_array_iterator.cc:184: FAILED: 5: CHECK( i == nval ) 5: with expansion: 5: 0 == 4 5: 5: [TileDB::utils] Error: (gzip_handle_error) Cannot compress with GZIP: deflateInit error: Z_STREAM_ERROR 5: [TileDB::Codec] Error: Could not compress with GZIP. 5: [TileDB::WriteState] Error: Cannot compress tile for /tmp/TileDBTest9tc7IW/array_iterator_test_ws/test_sparse_array_it_filter_reset/.__7e66b381-84bf-436b-932a-cad138365032281473382141984_1688334784896/ATTR_INT32.tdb. 5: ------------------------------------------------------------------------------- 5: Test sparse array iterator with filter with subarray 5: ------------------------------------------------------------------------------- 5: ./genomicsdb-tiledb/test/src/c_api/test_array_iterator.cc:227 5: ............................................................................... 5: 5: ./genomicsdb-tiledb/test/src/c_api/test_array_iterator.cc:106: FAILED: 5: CHECK( tiledb_array_write(tiledb_array, buffers, buffer_sizes) == 0 ) 5: with expansion: 5: -1 == 0 5: 5: [TileDB::utils] Error: (gzip_handle_error) Cannot compress with GZIP: deflateInit error: Z_STREAM_ERROR 5: [TileDB::Codec] Error: Could not compress with GZIP. 5: [TileDB::WriteState] Error: Cannot compress tile for /tmp/TileDBTest9tc7IW/array_iterator_test_ws/test_sparse_array_it_filter_reset/.__7e66b381-84bf-436b-932a-cad138365032281473382141984_1688334784896/ATTR_INT32.tdb. 5: ./genomicsdb-tiledb/test/src/c_api/test_array_iterator.cc:107: FAILED: 5: CHECK( tiledb_array_finalize(tiledb_array) == 0 ) 5: with expansion: 5: -1 == 0 5: 5: ./genomicsdb-tiledb/test/src/c_api/test_array_iterator.cc:184: FAILED: 5: CHECK( i == nval ) 5: with expansion: 5: 0 == 4 5: 5: ./genomicsdb-tiledb/test/src/c_api/test_array_iterator.cc:184: FAILED: 5: CHECK( i == nval ) 5: with expansion: 5: 0 == 1 5: 5: [TileDB::utils] Error: (gzip_handle_error) Cannot compress with GZIP: deflateInit error: Z_STREAM_ERROR 5: [TileDB::Codec] Error: Could not compress with GZIP. 5: [TileDB::WriteState] Error: Cannot compress tile for /tmp/TileDBTestcWIZ2d/array_iterator_test_ws/test_sparse_array_it_filter_with_empty_value/.__8beaeba5-e1cd-402f-9fb3-a00f104d7489281473382141984_1688334784897/ATTR_INT32.tdb. 5: ------------------------------------------------------------------------------- 5: Test sparse array iterator with filteri with empty value 5: ------------------------------------------------------------------------------- 5: ./genomicsdb-tiledb/test/src/c_api/test_array_iterator.cc:237 5: ............................................................................... 5: 5: ./genomicsdb-tiledb/test/src/c_api/test_array_iterator.cc:106: FAILED: 5: CHECK( tiledb_array_write(tiledb_array, buffers, buffer_sizes) == 0 ) 5: with expansion: 5: -1 == 0 5: 5: [TileDB::utils] Error: (gzip_handle_error) Cannot compress with GZIP: deflateInit error: Z_STREAM_ERROR 5: [TileDB::Codec] Error: Could not compress with GZIP. 5: [TileDB::WriteState] Error: Cannot compress tile for /tmp/TileDBTestcWIZ2d/array_iterator_test_ws/test_sparse_array_it_filter_with_empty_value/.__8beaeba5-e1cd-402f-9fb3-a00f104d7489281473382141984_1688334784897/ATTR_INT32.tdb. 5: ./genomicsdb-tiledb/test/src/c_api/test_array_iterator.cc:107: FAILED: 5: CHECK( tiledb_array_finalize(tiledb_array) == 0 ) 5: with expansion: 5: -1 == 0 5: 5: [TileDB::utils] Error: (gzip_handle_error) Cannot compress with GZIP: deflateInit error: Z_STREAM_ERROR 5: [TileDB::Codec] Error: Could not compress with GZIP. 5: [TileDB::WriteState] Error: Cannot compress tile for /tmp/TileDBTestcWIZ2d/array_iterator_test_ws/test_sparse_array_it_filter_with_empty_value/.__78263663-df68-463f-ac7b-fd5321190655281473382141984_1688334784897/ATTR_INT32.tdb. 5: ./genomicsdb-tiledb/test/src/c_api/test_array_iterator.cc:125: FAILED: 5: CHECK( tiledb_array_write(tiledb_array, buffers, buffer_sizes) == 0 ) 5: with expansion: 5: -1 == 0 5: 5: [TileDB::utils] Error: (gzip_handle_error) Cannot compress with GZIP: deflateInit error: Z_STREAM_ERROR 5: [TileDB::Codec] Error: Could not compress with GZIP. 5: [TileDB::WriteState] Error: Cannot compress tile for /tmp/TileDBTestcWIZ2d/array_iterator_test_ws/test_sparse_array_it_filter_with_empty_value/.__78263663-df68-463f-ac7b-fd5321190655281473382141984_1688334784897/ATTR_INT32.tdb. 5: ./genomicsdb-tiledb/test/src/c_api/test_array_iterator.cc:126: FAILED: 5: CHECK( tiledb_array_finalize(tiledb_array) == 0 ) 5: with expansion: 5: -1 == 0 5: 5: ./genomicsdb-tiledb/test/src/c_api/test_array_iterator.cc:184: FAILED: 5: CHECK( i == nval ) 5: with expansion: 5: 0 == 4 5: 5: =============================================================================== 5: test cases: 6 | 0 passed | 6 failed 5: assertions: 81 | 58 passed | 23 failed 5: 6: Test command: /<>/obj-aarch64-linux-gnu/genomicsdb-tiledb/test/test_array_schema_api 6: Working Directory: /<>/obj-aarch64-linux-gnu/genomicsdb-tiledb/test 6: Test timeout computed to be: 10000000 6: 6: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 6: test_array_schema_api is a Catch v2.13.10 host application. 6: Run with -? for options 6: 6: ------------------------------------------------------------------------------- 6: Test array schema backward compatibility 2 6: ------------------------------------------------------------------------------- 6: ./genomicsdb-tiledb/test/src/c_api/test_array_schema_api.cc:349 6: ............................................................................... 6: 6: ./genomicsdb-tiledb/test/src/c_api/test_array_schema_api.cc:368: FAILED: 6: CHECK( array_schema.compression_level_[1] == -1 ) 6: with expansion: 6: 255 == -1 6: 6: =============================================================================== 6: test cases: 13 | 12 passed | 1 failed 6: assertions: 335 | 334 passed | 1 failed 6: 19: Test command: /<>/genomicsdb-tiledb/examples/run_examples.sh 19: Working Directory: /<>/obj-aarch64-linux-gnu/genomicsdb-tiledb/examples 19: Test timeout computed to be: 10000000 19: Example 1: Running ./tiledb_workspace_group_create... 19: Example 1: Done running ./tiledb_workspace_group_create 19: Example 2: Running ./tiledb_ls_workspaces... 19: my_workspace 19: Example 2: Done running ./tiledb_ls_workspaces 19: Example 3: Running ./tiledb_array_create_dense... 19: Example 3: Done running ./tiledb_array_create_dense 19: Example 4: Running ./tiledb_array_create_sparse... 19: Example 4: Done running ./tiledb_array_create_sparse 19: Example 5: Running ./tiledb_array_primitive... 19: Array name: my_workspace/dense_arrays/my_array_A 19: Attributes: a1 a2 a3 19: The array is dense 19: Array name: my_workspace/sparse_arrays/my_array_B 19: Attributes: a1 a2 a3 19: The array is sparse 19: Example 5: Done running ./tiledb_array_primitive 19: Example 6: Running ./tiledb_array_write_dense_1... 19: [TileDB::utils] Error: (gzip_handle_error) Cannot compress with GZIP: deflateInit error: Z_STREAM_ERROR 19: [TileDB::Codec] Error: Could not compress with GZIP. 19: [TileDB::WriteState] Error: Cannot compress tile for /<>/obj-aarch64-linux-gnu/genomicsdb-tiledb/examples/my_workspace/dense_arrays/my_array_A/.__e3770d96-3113-4f9e-8089-24c0178c6400281473779200032_1688334785380/a1.tdb. 19: [TileDB::WriteState] Error: Cannot compress tile for /<>/obj-aarch64-linux-gnu/genomicsdb-tiledb/examples/my_workspace/dense_arrays/my_array_A/.__e3770d96-3113-4f9e-8089-24c0178c6400281473779200032_1688334785380/a1.tdb[Examples::./genomicsdb-tiledb/examples/src/tiledb_array_write_dense_1.cc] Runtime Error. 19: 19: Exit Status=255. Quitting execution of run_examples.sh 20: Test command: /<>/obj-aarch64-linux-gnu/src/test/cpp/ctests "-d" "yes" 20: Working Directory: /<>/obj-aarch64-linux-gnu/src/test/cpp 20: Test timeout computed to be: 10000000 20: 21:53:05.118 DEBUG NativeGenomicsDB - pid=3527814 tid=3527814 json file /<>/obj-aarch64-linux-gnu/src/test/cpp/inputs/loader.json does not seem to be serialized protobuf - error:INVALID_ARGUMENT:(column_partitions[0].begin): invalid value 0 for type type.googleapis.com/GenomicsDBColumn 20: 21:53:05.119 WARN NativeGenomicsDB - pid=3527814 tid=3527814 Could not deserialize loader file /<>/obj-aarch64-linux-gnu/src/test/cpp/inputs/loader.json as protobuf. Trying to parse as a regular JSON file instead 20: 21:53:05.123 DEBUG NativeGenomicsDB - pid=3527814 tid=3527814 json file inputs/vid.json does not seem to be serialized protobuf - error:INVALID_ARGUMENT:(contigs): missing field name 20: 0.015 s: Check loader configuration with json file 20: 21:53:05.129 DEBUG NativeGenomicsDB - pid=3527814 tid=3527814 json file /<>/obj-aarch64-linux-gnu/src/test/cpp/inputs/loader.json does not seem to be serialized protobuf - error:INVALID_ARGUMENT:(column_partitions[0].begin): invalid value 0 for type type.googleapis.com/GenomicsDBColumn 20: 21:53:05.130 WARN NativeGenomicsDB - pid=3527814 tid=3527814 Could not deserialize loader file /<>/obj-aarch64-linux-gnu/src/test/cpp/inputs/loader.json as protobuf. Trying to parse as a regular JSON file instead 20: 21:53:05.131 DEBUG NativeGenomicsDB - pid=3527814 tid=3527814 json file inputs/vid.json does not seem to be serialized protobuf - error:INVALID_ARGUMENT:(contigs): missing field name 20: 0.003 s: Check configuration with json file 20: 21:53:05.135 DEBUG NativeGenomicsDB - pid=3527814 tid=3527814 json file /<>/obj-aarch64-linux-gnu/src/test/cpp/inputs/loader.json does not seem to be serialized protobuf - error:INVALID_ARGUMENT:(column_partitions[0].begin): invalid value 0 for type type.googleapis.com/GenomicsDBColumn 20: 21:53:05.137 WARN NativeGenomicsDB - pid=3527814 tid=3527814 Could not deserialize loader file /<>/obj-aarch64-linux-gnu/src/test/cpp/inputs/loader.json as protobuf. Trying to parse as a regular JSON file instead 20: 21:53:05.139 DEBUG NativeGenomicsDB - pid=3527814 tid=3527814 json file inputs/vid.json does not seem to be serialized protobuf - error:INVALID_ARGUMENT:(contigs): missing field name 20: 0.009 s: Check configuration with json file and shared posifs optim 20: 21:53:05.144 DEBUG NativeGenomicsDB - pid=3527814 tid=3527814 json file /<>/obj-aarch64-linux-gnu/src/test/cpp/inputs/loader.json does not seem to be serialized protobuf - error:INVALID_ARGUMENT:(column_partitions[0].begin): invalid value 0 for type type.googleapis.com/GenomicsDBColumn 20: 21:53:05.145 WARN NativeGenomicsDB - pid=3527814 tid=3527814 Could not deserialize loader file /<>/obj-aarch64-linux-gnu/src/test/cpp/inputs/loader.json as protobuf. Trying to parse as a regular JSON file instead 20: 21:53:05.147 DEBUG NativeGenomicsDB - pid=3527814 tid=3527814 json file inputs/vid.json does not seem to be serialized protobuf - error:INVALID_ARGUMENT:(contigs): missing field name 20: 0.007 s: Compare configuration with json file and string 20: 0.000 s: genotype_ordering haploid_5_alleles 20: 0.000 s: genotype_ordering diploid_2_alleles 20: 0.000 s: genotype_ordering diploid_3_alleles 20: 0.000 s: genotype_ordering diploid_4_alleles 20: 0.000 s: genotype_ordering triploid_2_alleles 20: 0.000 s: genotype_ordering triploid_3_alleles 20: 0.000 s: genotype_ordering triploid_4_alleles 20: 0.000 s: genotype_ordering triploid_4_alleles_reordered_alleles 20: 0.000 s: genotype_ordering triploid_4_alleles_reordered_alleles_without_NON_REF 20: 0.001 s: multid_vector 2D_test 20: 0.000 s: multid_vector 3D_test 20: 0.000 s: multid_vector 2D_test_with_missing_values 20: 0.000 s: multid_vector 2D_float_test 20: 0.000 s: multid_vector 2D_tuple_test 20: 0.000 s: pb_query_config_test 20: 0.004 s: pb_vid_mapping_test 20: 21:53:05.178 INFO NativeGenomicsDB - pid=3527814 tid=3527814 Contig/chromosome 2 begins at TileDB column 249250601 and intersects with contig/chromosome 1 that spans columns [0, 249250620] 20: 0.004 s: pb_overlapping_vid_mapping_test 20: 21:53:05.180 INFO NativeGenomicsDB - pid=3527814 tid=3527814 This is a test 20: 21:53:05.180 WARN NativeGenomicsDB - pid=3527814 tid=3527814 This is a test 20: 21:53:05.180 ERROR NativeGenomicsDB - pid=3527814 tid=3527814 This is a test 20: 21:53:05.181 DEBUG NativeGenomicsDB - pid=3527814 tid=3527814 This is a test 20: This is a test 20: This is a test 20: This is a test 20: This is a test 20: This is a once_only test 20: Trying format Hello1 Hello2 20: 0.005 s: test logger 20: 21:53:05.186 INFO NativeGenomicsDB - pid=3527814 tid=3527814 i=1 str=This is a test 20: 21:53:05.186 WARN NativeGenomicsDB - pid=3527814 tid=3527814 i=2 str=This is a test 20: 21:53:05.186 ERROR NativeGenomicsDB - pid=3527814 tid=3527814 i=3 str=This is a test 20: 21:53:05.187 DEBUG NativeGenomicsDB - pid=3527814 tid=3527814 i=4 str=This is a test 20: 21:53:05.187 INFO NativeGenomicsDB - pid=3527814 tid=3527814 i=6 str=This is a once_only test 20: 21:53:05.187 WARN NativeGenomicsDB - pid=3527814 tid=3527814 i=7 str=This is a once_only test 20: 0.001 s: test logger format 20: 21:53:05.187 ERROR NativeGenomicsDB - pid=3527814 tid=3527814 i=1 str=This is a test 20: 21:53:05.187 ERROR NativeGenomicsDB - pid=3527814 tid=3527814 Test Exception 20: 21:53:05.187 ERROR NativeGenomicsDB - pid=3527814 tid=3527814 Test Exception 20: 21:53:05.187 ERROR NativeGenomicsDB - pid=3527814 tid=3527814 Test Exception 20: 21:53:05.188 ERROR NativeGenomicsDB - pid=3527814 tid=3527814 Native Stack Trace: 20: /<>/obj-aarch64-linux-gnu/src/test/cpp/ctests(+0xd2f74) [0xaaaae30a2f74] 20: /<>/obj-aarch64-linux-gnu/src/test/cpp/ctests(+0xc70c4) [0xaaaae30970c4] 20: /<>/obj-aarch64-linux-gnu/src/test/cpp/ctests(+0x4e0e8) [0xaaaae301e0e8] 20: /<>/obj-aarch64-linux-gnu/src/test/cpp/ctests(+0x6688c) [0xaaaae303688c] 20: /<>/obj-aarch64-linux-gnu/src/test/cpp/ctests(+0x6cde4) [0xaaaae303cde4] 20: /<>/obj-aarch64-linux-gnu/src/test/cpp/ctests(+0x6ea70) [0xaaaae303ea70] 20: /<>/obj-aarch64-linux-gnu/src/test/cpp/ctests(+0x6f0c0) [0xaaaae303f0c0] 20: /<>/obj-aarch64-linux-gnu/src/test/cpp/ctests(+0x3fee0) [0xaaaae300fee0] 20: /lib/aarch64-linux-gnu/libc.so.6(+0x27780) [0xffffab0e7780] 20: 21:53:05.188 ERROR NativeGenomicsDB - pid=3527814 tid=3527814 Test Exception 20: 21:53:05.188 ERROR NativeGenomicsDB - pid=3527814 tid=3527814 Native Stack Trace: 20: /<>/obj-aarch64-linux-gnu/src/test/cpp/ctests(+0xd2f74) [0xaaaae30a2f74] 20: /<>/obj-aarch64-linux-gnu/src/test/cpp/ctests(+0xc70d4) [0xaaaae30970d4] 20: /<>/obj-aarch64-linux-gnu/src/test/cpp/ctests(+0x4e0e8) [0xaaaae301e0e8] 20: /<>/obj-aarch64-linux-gnu/src/test/cpp/ctests(+0x6688c) [0xaaaae303688c] 20: /<>/obj-aarch64-linux-gnu/src/test/cpp/ctests(+0x6cde4) [0xaaaae303cde4] 20: /<>/obj-aarch64-linux-gnu/src/test/cpp/ctests(+0x6ea70) [0xaaaae303ea70] 20: /<>/obj-aarch64-linux-gnu/src/test/cpp/ctests(+0x6f0c0) [0xaaaae303f0c0] 20: /<>/obj-aarch64-linux-gnu/src/test/cpp/ctests(+0x3fee0) [0xaaaae300fee0] 20: /lib/aarch64-linux-gnu/libc.so.6(+0x27780) [0xffffab0e7780] 20: 0.003 s: test logger exceptions 20: /<>/obj-aarch64-linux-gnu/src/test/cpp/ctests(+0xd2f74) [0xaaaae30a2f74] 20: /<>/obj-aarch64-linux-gnu/src/test/cpp/ctests(+0xc9c98) [0xaaaae3099c98] 20: /<>/obj-aarch64-linux-gnu/src/test/cpp/ctests(+0x4e0e8) [0xaaaae301e0e8] 20: /<>/obj-aarch64-linux-gnu/src/test/cpp/ctests(+0x6688c) [0xaaaae303688c] 20: /<>/obj-aarch64-linux-gnu/src/test/cpp/ctests(+0x6cde4) [0xaaaae303cde4] 20: /<>/obj-aarch64-linux-gnu/src/test/cpp/ctests(+0x6ea70) [0xaaaae303ea70] 20: /<>/obj-aarch64-linux-gnu/src/test/cpp/ctests(+0x6f0c0) [0xaaaae303f0c0] 20: /<>/obj-aarch64-linux-gnu/src/test/cpp/ctests(+0x3fee0) [0xaaaae300fee0] 20: /lib/aarch64-linux-gnu/libc.so.6(+0x27780) [0xffffab0e7780] 20: /<>/obj-aarch64-linux-gnu/src/test/cpp/ctests(+0xd2f74) [0xaaaae30a2f74] 20: /<>/obj-aarch64-linux-gnu/src/test/cpp/ctests(+0xc9d28) [0xaaaae3099d28] 20: /<>/obj-aarch64-linux-gnu/src/test/cpp/ctests(+0x4e0e8) [0xaaaae301e0e8] 20: /<>/obj-aarch64-linux-gnu/src/test/cpp/ctests(+0x6688c) [0xaaaae303688c] 20: /<>/obj-aarch64-linux-gnu/src/test/cpp/ctests(+0x6cde4) [0xaaaae303cde4] 20: /<>/obj-aarch64-linux-gnu/src/test/cpp/ctests(+0x6ea70) [0xaaaae303ea70] 20: /<>/obj-aarch64-linux-gnu/src/test/cpp/ctests(+0x6f0c0) [0xaaaae303f0c0] 20: /<>/obj-aarch64-linux-gnu/src/test/cpp/ctests(+0x3fee0) [0xaaaae300fee0] 20: /lib/aarch64-linux-gnu/libc.so.6(+0x27780) [0xffffab0e7780] 20: This is a test 20: 0.004 s: test explicit logger 20: 21:53:05.196 DEBUG NativeGenomicsDB - pid=3527814 tid=3527814 json file /<>/obj-aarch64-linux-gnu/src/test/cpp/inputs/loader.json does not seem to be serialized protobuf - error:INVALID_ARGUMENT:(column_partitions[0].begin): invalid value 0 for type type.googleapis.com/GenomicsDBColumn 20: 21:53:05.197 WARN NativeGenomicsDB - pid=3527814 tid=3527814 Could not deserialize loader file /<>/obj-aarch64-linux-gnu/src/test/cpp/inputs/loader.json as protobuf. Trying to parse as a regular JSON file instead 20: 21:53:05.202 DEBUG NativeGenomicsDB - pid=3527814 tid=3527814 json file inputs/vid.json does not seem to be serialized protobuf - error:INVALID_ARGUMENT:(contigs): missing field name 20: 0.055 s: gvcf iterator 20: 21:53:05.253 DEBUG NativeGenomicsDB - pid=3527814 tid=3527814 json file /<>/obj-aarch64-linux-gnu/src/test/cpp/inputs/loader.json does not seem to be serialized protobuf - error:INVALID_ARGUMENT:(column_partitions[0].begin): invalid value 0 for type type.googleapis.com/GenomicsDBColumn 20: 21:53:05.254 WARN NativeGenomicsDB - pid=3527814 tid=3527814 Could not deserialize loader file /<>/obj-aarch64-linux-gnu/src/test/cpp/inputs/loader.json as protobuf. Trying to parse as a regular JSON file instead 20: 21:53:05.255 DEBUG NativeGenomicsDB - pid=3527814 tid=3527814 json file inputs/vid.json does not seem to be serialized protobuf - error:INVALID_ARGUMENT:(contigs): missing field name 20: 0.043 s: columnar_gvcf_iterator_test 20: [TileDB] Error: No TileDB support for URL=dfdfd://ddd. 20: 21:53:05.297 ERROR NativeGenomicsDB - pid=3527814 tid=3527814 htslib_plugin could not open file dfdfd://ddd No TileDB support for URL=dfdfd://ddd 20: [TileDB::FileSystem] Error: (write_to_file) Cannot write to file; File opening error path=/dfdfd/dfdfd/non-existent-file errno=2(No such file or directory) 20: 21:53:05.299 ERROR NativeGenomicsDB - pid=3527814 tid=3527814 hts_plugin read /dfdfd/dfdfd/non-existent-file error [TileDB::FileSystem] Error: (write_to_file) Cannot write to file; File opening error path=/dfdfd/dfdfd/non-existent-file errno=2(No such file or directory) 20: [TileDB::FileSystem] Error: (read_from_file) Cannot read from file; File opening error path=non-existent-file errno=2(No such file or directory) 20: 21:53:05.301 ERROR NativeGenomicsDB - pid=3527814 tid=3527814 hts_plugin read non-existent-file error [TileDB::FileSystem] Error: (read_from_file) Cannot read from file; File opening error path=non-existent-file errno=2(No such file or directory) 20: 0.008 s: test htslib plugin 20: 0.000 s: alleles_combiner 20: 0.000 s: gt_remapper 20: 20: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 20: ctests is a Catch v2.13.10 host application. 20: Run with -? for options 20: 20: ------------------------------------------------------------------------------- 20: test bgen 20: ------------------------------------------------------------------------------- 20: ./src/test/cpp/src/test_bgen.cc:11 20: ............................................................................... 20: 20: ./src/test/cpp/src/test_bgen.cc:11: FAILED: 20: due to unexpected exception with message: 20: Direct - first BGEN test output did not match reference output 20: 20: 0.000 s: test bgen 20: 21:53:05.961 ERROR NativeGenomicsDB - pid=3527814 tid=3527814 VariantStorageManagerException exception : Could not open workspace 20: 21:53:05.961 ERROR NativeGenomicsDB - pid=3527814 tid=3527814 Could not open array non_existent_array in workspace /tmp/GenomicsDB7P2Z05/ws 20: 21:53:05.961 ERROR NativeGenomicsDB - pid=3527814 tid=3527814 VariantQueryProcessorException : Could not open array non_existent_array at workspace /tmp/GenomicsDB7P2Z05/ws 20: 0.001 s: constructor 20: 21:53:05.964 DEBUG NativeGenomicsDB - pid=3527814 tid=3527814 json file /<>/obj-aarch64-linux-gnu/src/test/cpp/inputs/loader.json does not seem to be serialized protobuf - error:INVALID_ARGUMENT:(column_partitions[0].begin): invalid value 0 for type type.googleapis.com/GenomicsDBColumn 20: 21:53:05.966 WARN NativeGenomicsDB - pid=3527814 tid=3527814 Could not deserialize loader file /<>/obj-aarch64-linux-gnu/src/test/cpp/inputs/loader.json as protobuf. Trying to parse as a regular JSON file instead 20: 21:53:05.967 DEBUG NativeGenomicsDB - pid=3527814 tid=3527814 json file inputs/vid.json does not seem to be serialized protobuf - error:INVALID_ARGUMENT:(contigs): missing field name 20: 21:53:05.979 ERROR NativeGenomicsDB - pid=3527814 tid=3527814 Invalid query attribute : non_existent_attribute 20: 0.019 s: test basic queries 20: =============================================================================== 20: test cases: 33 | 32 passed | 1 failed 20: assertions: 1434 | 1433 passed | 1 failed 20: 20: 21:53:05.982 DEBUG NativeGenomicsDB - pid=3527814 tid=3527814 Deleting htslib_fs_adapter tiledb_ctx_map 22: Test command: /<>/tests/test_tools.sh "/<>/tests/inputs/vcfs" "/<>/obj-aarch64-linux-gnu/tools" "1" 22: Working Directory: /<>/obj-aarch64-linux-gnu/tools 22: Test timeout computed to be: 10000000 22: 21:53:06.657 INFO NativeGenomicsDB - pid=3527929 tid=3527929 Reading 1 vcfs 22: 21:53:06.660 INFO NativeGenomicsDB - pid=3527929 tid=3527929 [STAGE 1 / 3] Reading 1 / 1 = 100.00% 22: 21:53:06.780 INFO NativeGenomicsDB - pid=3527929 tid=3527929 [STAGE 2 / 3] read_next_batch 0 / 1 = 0.00% 22: 21:53:06.782 INFO NativeGenomicsDB - pid=3527929 tid=3527929 [STAGE 3 / 3] progress: 12140 / 249250620 = 0.00% 22: [TileDB::utils] Error: (gzip_handle_error) Cannot compress with GZIP: deflateInit error: Z_STREAM_ERROR 22: [TileDB::Codec] Error: Could not compress with GZIP. 22: [TileDB::WriteState] Error: Cannot compress tile for /tmp/test_tools-3MWKYDtPtg/ws_27716/1$1$249250621/.__a6fea7aa-d4a5-4028-9541-375ffb29be3a281473138085952_1688334786747/END.tdb. 22: 21:53:06.784 ERROR NativeGenomicsDB - pid=3527929 tid=3527929 VariantStorageManagerException exception : Error while finalizing GenomicsDB array 1$1$249250621 : [TileDB::WriteState] Error: Cannot compress tile for /tmp/test_tools-3MWKYDtPtg/ws_27716/1$1$249250621/.__a6fea7aa-d4a5-4028-9541-375ffb29be3a281473138085952_1688334786747/END.tdb 22: command 'vcf2genomicsdb --progress /tmp/test_tools-3MWKYDtPtg/ws_27716/loader.json' returned 139 unexpectedly 77% tests passed, 5 tests failed out of 22 Total Test time (real) = 5.37 sec The following tests FAILED: 5 - test_array_iterator (Failed) 6 - test_array_schema_api (Failed) 19 - ci_tests (Failed) 20 - ctests (Failed) 22 - tools_tests (Failed) ```

https://buildd.debian.org/status/fetch.php?pkg=genomicsdb&arch=arm64&ver=1.5.0-1&stamp=1688334799&raw=0