Galera-4 multi-platform build support on Debian

ottok commented 4 years ago

I created this as a meta-issue to track galera-4 build status on the Debian platforms. All official platforms build OK, so there is nothing urgent to do here.

Status when this issue was filed:

The latest status can always be checked at https://buildd.debian.org/status/package.php?p=galera-4

Failures:

hurd-i386 fails completely due to missing dependency. Hurd isn't properly maintained anymore, so it can be just ignored.

hppa seem to be due to failing test suite:

Running suite(s): gu::RecordSet
66%: Checks: 6, Failures: 0, Errors: 2
galerautils/tests/gu_rset_test.cpp:337:E:RecordSet v1:ver1:0: (after this point) Test timeout expired
galerautils/tests/gu_rset_test.cpp:395:E:RecordSet v2:ver2:0: (after this point) Test timeout expired

sparc64 also fails in test suite:

Running suite(s): DataSet
0%: Checks: 2, Failures: 0, Errors: 2
galera/tests/data_set_check.cpp:243:E:DataSet:ver1:0: (after this point) Received signal 10 (Bus error)
galera/tests/data_set_check.cpp:143:E:DataSet:ver2:0: (after this point) Received signal 10 (Bus error)
Running suite(s): KeySet
0%: Checks: 4, Failures: 0, Errors: 4
galera/tests/key_set_check.cpp:54:E:KeySet:ver1_3:0: (after this point) Received signal 10 (Bus error)
galera/tests/key_set_check.cpp:235:E:KeySet:ver2_3:0: (after this point) Received signal 10 (Bus error)
galera/tests/key_set_check.cpp:235:E:KeySet:ver2_4:0: (after this point) Received signal 10 (Bus error)
galera/tests/key_set_check.cpp:235:E:KeySet:ver2_5:0: (after this point) Received signal 10 (Bus error)
Running suite(s): WriteSet
60%: Checks: 5, Failures: 0, Errors: 2
galera/tests/write_set_ng_check.cpp:83:E:WriteSet basic:ver3_basic_rsv1:0: (after this point) Received signal 10 (Bus error)
galera/tests/write_set_ng_check.cpp:307:E:WriteSet annotation:ver3_annotation_rsv1:0: (after this point) Received signal 10 (Bus error)

ia64 and powerpc builds return a failure code, but the test suite did pass and it is unclear to me why they failed. Logs:
- ia64: https://buildd.debian.org/status/fetch.php?pkg=galera-4&arch=ia64&ver=26.4.3-1&stamp=1577391125&raw=0
- powerpc: https://buildd.debian.org/status/fetch.php?pkg=galera-4&arch=powerpc&ver=26.4.3-1&stamp=1577390834&raw=0

Warning in many build logs:

scons: Configure: Checking for SSL_CTX_set_ecdh_auto() ... 
.sconf_temp/conftest_14.cpp <-
  |
  |#include <openssl/ssl.h>
  |int main() { SSL_CTX* ctx=NULL; return !SSL_CTX_set_ecdh_auto(ctx, 1); }
  |
g++ -o .sconf_temp/conftest_14.o -c -std=c++11 -pipe -Wno-long-long -Wno-deprecated -g -O2 -fdebug-prefix-map=/<<PKGBUILDDIR>>=. -Wformat -Werror=format-security -g -O3 -DNDEBUG -pthread -fPIC -Wall -Wextra -Wno-unused-parameter -Wdate-time -D_FORTIFY_SOURCE=2 -D_XOPEN_SOURCE=600 -DHAVE_COMMON_H -DGALERA_USE_GU_NETWORK -DHAVE_BYTESWAP_H -DHAVE_ENDIAN_H -DHAVE_EXECINFO_H -DHAVE_STD_ARRAY -DHAVE_BOOST_SHARED_PTR_HPP -DHAVE_STD_UNORDERED_MAP -DBOOST_DATE_TIME_POSIX_TIME_STD_CONFIG=1 -DHAVE_ASIO_HPP -Iasio -Iwsrep/src .sconf_temp/conftest_14.cpp
.sconf_temp/conftest_14.cpp: In function 'int main()':
.sconf_temp/conftest_14.cpp:3:23: warning: unused variable 'ctx' [-Wunused-variable]
    3 | int main() { SSL_CTX* ctx=NULL; return !SSL_CTX_set_ecdh_auto(ctx, 1); }
      |

ottok commented 4 years ago

Current status:

= hurd-i386 has been fixed.

Remaining ones:

hppa

33%: Checks: 3, Failures: 1, Errors: 1
galera/tests/saved_state_check.cpp:47:E:saved_state:test_unsafe:0: (after this point) Test timeout expired
galera/tests/saved_state_check.cpp:162:F:saved_state:test_corrupt:0: Failure 'uuid == WSREP_UUID_UNDEFINED' occurred
Running suite(s): Defaults
It may not be safe to bootstrap the cluster from this node. It was not the last one to leave the cluster and may not contain all the updates. To force cluster bootstrap with this node, edit the grastate.dat file manually and set safe_to_bootstrap to 1 .
0%: Checks: 1, Failures: 1, Errors: 0
galera/tests/defaults_check.cpp:277:F:defaults:defaults:0: connect() returned 7
Total tests failed: 3

ia64

Checking dynamic symbols for 'libgalera_smm.so'...
00000000000f1a80 l    DF .text  0000000000000010              _ZN4asio6detail10service_idINS0_13epoll_reactorEED1Ev
0000000000305980 l    DF .text  00000000000000d0              _ZN4asio3ssl6detail6engine9do_acceptEPvm
0000000000344cc0 l    DF .text  00000000000007c0              _ZN4asio6detail18completion_handlerIN5gcomm22AsioPostForSendHandlerEE11do_completeEPNS0_15task_io_serviceEPNS0_25task_io_service_operationERKSt10error_codem
00000000000f1dc0 l    DF .text  0000000000000060              _ZN4asio5error6detail12ssl_categoryD1Ev
0000000000342d80 l    DF .text  0000000000000360              _ZN4asio6detail23reactive_socket_recv_opINS_17mutable_buffers_1ENS_3ssl6detail5io_opINS_19basic_stream_socketINS_2ip3tcpENS_21stream_socket_serviceIS8_EEEENS4_7read_opINS0_17consuming_buffersINS_14mutable_bufferESt5arrayISE_Lm1EEEEEENS0_7read_opINS3_6streamISB_EESG_N5boost3_bi6bind_tImNSM_4_mfi3mf2ImN5gcomm13AsioTcpSocketERKSt10error_codemEENSN_5list3INSN_5valueINSM_10shared_ptrISS_EEEEPFNSM_3argILi1EEEvEPFNS12_ILi2EEEvEEEEENSO_IvNSQ_IvSS_SV_mEES19_EEEEEEE11do_completeEPNS0_15task_io_serviceEPNS0_25task_io_service_operationESV_m
000000000032f380 l    DF .text  0000000000000470              _ZN4asio6detail26reactive_socket_connect_opIN5boost3_bi6bind_tIvNS2_4_mfi3mf1IvN5gcomm13AsioTcpSocketERKSt10error_codeEENS3_5list2INS3_5valueINS2_10shared_ptrIS8_EEEEPFNS2_3argILi1EEEvEEEEEE11do_completeEPNS0_15task_io_serviceEPNS0_25task_io_service_operationESB_m
0000000000311480 l    DF .text  00000000000006d0              _ZN4asio6detail28reactive_socket_send_op_baseINS_17mutable_buffers_1EE10do_performEPNS0_10reactor_opE
0000000000345480 l    DF .text  00000000000004f0              _ZN4asio6detail23reactive_socket_recv_opINS_17mutable_buffers_1ENS_3ssl6detail5io_opINS_19basic_stream_socketINS_2ip3tcpENS_21stream_socket_serviceIS8_EEEENS4_8write_opISt5arrayINS_12const_bufferELm2EEEENS0_8write_opINS3_6streamISB_EESF_NS0_14transfer_all_tEN5boost3_bi6bind_tIvNSL_4_mfi3mf2IvN5gcomm13AsioTcpSocketERKSt10error_codemEENSM_5list3INSM_5valueINSL_10shared_ptrISR_EEEEPFNSL_3argILi1EEEvEPFNS11_ILi2EEEvEEEEEEEEEE11do_completeEPNS0_15task_io_serviceEPNS0_25task_io_service_operationESU_m
0000000000353d00 l    DF .text  0000000000000010              _ZN4asio6detail10service_idINS_2ip16resolver_serviceINS2_3udpEEEED1Ev
000000000031ab40 l    DF .text  0000000000000760              _ZN4asio6detail13epoll_reactor16descriptor_state11do_completeEPNS0_15task_io_serviceEPNS0_25task_io_service_operationERKSt10error_codem
0000000000303c80 l    DF .text  0000000000000010              _ZN4asio6detail10service_idINS_2ip16resolver_serviceINS2_3tcpEEEED1Ev
0000000000353d40 l    DF .text  0000000000000010              _ZN4asio6detail10service_idINS_23datagram_socket_serviceINS_2ip3udpEEEED1Ev
0000000000303cc0 l    DF .text  0000000000000010              _ZN4asio6detail10service_idINS_23socket_acceptor_serviceINS_2ip3tcpEEEED1Ev
00000000003424c0 l    DF .text  00000000000008a0              _ZN4asio6detail23reactive_socket_send_opINS_17mutable_buffers_1ENS0_8write_opINS_19basic_stream_socketINS_2ip3tcpENS_21stream_socket_serviceIS6_EEEES2_NS0_14transfer_all_tENS_3ssl6detail5io_opIS9_NSC_7read_opINS0_17consuming_buffersINS_14mutable_bufferESt5arrayISG_Lm1EEEEEENS0_7read_opINSB_6streamIS9_EESI_N5boost3_bi6bind_tImNSO_4_mfi3mf2ImN5gcomm13AsioTcpSocketERKSt10error_codemEENSP_5list3INSP_5valueINSO_10shared_ptrISU_EEEEPFNSO_3argILi1EEEvEPFNS14_ILi2EEEvEEEEENSQ_IvNSS_IvSU_SX_mEES1B_EEEEEEEEE11do_completeEPNS0_15task_io_serviceEPNS0_25task_io_service_operationESX_m
0000000000337b00 l    DF .text  00000000000011f0              _ZN4asio6detail23reactive_socket_recv_opINS0_17consuming_buffersINS_14mutable_bufferESt5arrayIS3_Lm1EEEENS0_7read_opINS_19basic_stream_socketINS_2ip3tcpENS_21stream_socket_serviceISA_EEEES5_N5boost3_bi6bind_tImNSE_4_mfi3mf2ImN5gcomm13AsioTcpSocketERKSt10error_codemEENSF_5list3INSF_5valueINSE_10shared_ptrISK_EEEEPFNSE_3argILi1EEEvEPFNSU_ILi2EEEvEEEEENSG_IvNSI_IvSK_SN_mEES11_EEEEE11do_completeEPNS0_15task_io_serviceEPNS0_25task_io_service_operationESN_m
00000000000f2e00 l    DF .text  0000000000000160              _ZNSt10shared_ptrIN4asio3ssl6detail17openssl_init_base7do_initEED1Ev
0000000000305f80 l    DF .text  0000000000000080              _ZN4asio3ssl6detail6engine8do_writeEPvm
0000000000343100 l    DF .text  0000000000000350              _ZN4asio6detail12wait_handlerINS_3ssl6detail5io_opINS_19basic_stream_socketINS_2ip3tcpENS_21stream_socket_serviceIS7_EEEENS3_7read_opINS0_17consuming_buffersINS_14mutable_bufferESt5arrayISD_Lm1EEEEEENS0_7read_opINS2_6streamISA_EESF_N5boost3_bi6bind_tImNSL_4_mfi3mf2ImN5gcomm13AsioTcpSocketERKSt10error_codemEENSM_5list3INSM_5valueINSL_10shared_ptrISR_EEEEPFNSL_3argILi1EEEvEPFNS11_ILi2EEEvEEEEENSN_IvNSP_IvSR_SU_mEES18_EEEEEEE11do_completeEPNS0_15task_io_serviceEPNS0_25task_io_service_operationESU_m
0000000000331780 l    DF .text  0000000000000650              _ZN4asio6detail25reactive_socket_accept_opINS_12basic_socketINS_2ip3tcpENS_21stream_socket_serviceIS4_EEEES4_N5boost3_bi6bind_tIvNS8_4_mfi3mf2IvN5gcomm15AsioTcpAcceptorENS8_10shared_ptrINSD_6SocketEEERKSt10error_codeEENS9_5list3INS9_5valueIPSE_EENSN_ISH_EEPFNS8_3argILi1EEEvEEEEEE11do_completeEPNS0_15task_io_serviceEPNS0_25task_io_service_operationESK_m
00000000000f21c0 l    DF .text  0000000000000060              _ZN4asio3ssl5error6detail15stream_categoryD1Ev
0000000000305a80 l    DF .text  0000000000000050              _ZN4asio3ssl6detail6engine10do_connectEPvm
00000000000f2f80 l    DF .text  0000000000000160              _ZN4asio3ssl6detail12openssl_initILb1EED1Ev
0000000000345980 l    DF .text  00000000000005d0              _ZN4asio6detail12wait_handlerINS_3ssl6detail5io_opINS_19basic_stream_socketINS_2ip3tcpENS_21stream_socket_serviceIS7_EEEENS3_8write_opISt5arrayINS_12const_bufferELm2EEEENS0_8write_opINS2_6streamISA_EESE_NS0_14transfer_all_tEN5boost3_bi6bind_tIvNSK_4_mfi3mf2IvN5gcomm13AsioTcpSocketERKSt10error_codemEENSL_5list3INSL_5valueINSK_10shared_ptrISQ_EEEEPFNSK_3argILi1EEEvEPFNS10_ILi2EEEvEEEEEEEEEE11do_completeEPNS0_15task_io_serviceEPNS0_25task_io_service_operationEST_m
0000000000312c80 l    DF .text  00000000000007f0              _ZN4asio6detail28reactive_socket_recv_op_baseINS_17mutable_buffers_1EE10do_performEPNS0_10reactor_opE
0000000000324c40 l    DF .text  0000000000000310              _ZN4asio6detail16service_registry6createINS_23socket_acceptor_serviceINS_2ip3tcpEEEEEPNS_10io_service7serviceERS7_
00000000000f2bc0 l    DF .text  0000000000000230              _ZN4asio3ssl7context26password_callback_functionEPciiPv
0000000000324900 l    DF .text  0000000000000310              _ZN4asio6detail16service_registry6createINS_21stream_socket_serviceINS_2ip3tcpEEEEEPNS_10io_service7serviceERS7_
000000000033d640 l    DF .text  00000000000004f0              _ZN4asio6detail12wait_handlerINS_3ssl6detail5io_opINS_19basic_stream_socketINS_2ip3tcpENS_21stream_socket_serviceIS7_EEEENS3_12handshake_opEN5boost3_bi6bind_tIvNSC_4_mfi3mf1IvN5gcomm13AsioTcpSocketERKSt10error_codeEENSD_5list2INSD_5valueINSC_10shared_ptrISI_EEEEPFNSC_3argILi1EEEvEEEEEEEE11do_completeEPNS0_15task_io_serviceEPNS0_25task_io_service_operationESL_m
000000000033a2c0 l    DF .text  00000000000004e0              _ZN4asio6detail23reactive_socket_send_opISt5arrayINS_12const_bufferELm2EENS0_8write_opINS_19basic_stream_socketINS_2ip3tcpENS_21stream_socket_serviceIS8_EEEES4_NS0_14transfer_all_tEN5boost3_bi6bind_tIvNSD_4_mfi3mf2IvN5gcomm13AsioTcpSocketERKSt10error_codemEENSE_5list3INSE_5valueINSD_10shared_ptrISJ_EEEEPFNSD_3argILi1EEEvEPFNST_ILi2EEEvEEEEEEEE11do_completeEPNS0_15task_io_serviceEPNS0_25task_io_service_operationESM_m
0000000000305f00 l    DF .text  0000000000000080              _ZN4asio3ssl6detail6engine7do_readEPvm
0000000000361ec0 l    DF .text  0000000000000370              _ZN4asio6detail12wait_handlerIN5boost3_bi6bind_tIvNS2_4_mfi3mf1IvN5gcomm12AsioProtonetERKSt10error_codeEENS3_5list2INS3_5valueIPS8_EEPFNS2_3argILi1EEEvEEEEEE11do_completeEPNS0_15task_io_serviceEPNS0_25task_io_service_operationESB_m
00000000000f20c0 l    DF .text  0000000000000060              _ZN4asio5error6detail13misc_categoryD1Ev
0000000000345f80 l    DF .text  0000000000000b20              _ZN4asio6detail23reactive_socket_send_opINS_17mutable_buffers_1ENS0_8write_opINS_19basic_stream_socketINS_2ip3tcpENS_21stream_socket_serviceIS6_EEEES2_NS0_14transfer_all_tENS_3ssl6detail5io_opIS9_NSC_8write_opISt5arrayINS_12const_bufferELm2EEEENS3_INSB_6streamIS9_EESH_SA_N5boost3_bi6bind_tIvNSL_4_mfi3mf2IvN5gcomm13AsioTcpSocketERKSt10error_codemEENSM_5list3INSM_5valueINSL_10shared_ptrISR_EEEEPFNSL_3argILi1EEEvEPFNS11_ILi2EEEvEEEEEEEEEEEE11do_completeEPNS0_15task_io_serviceEPNS0_25task_io_service_operationESU_m
0000000000303880 l    DF .text  0000000000000010              _ZN4asio12placeholders17bytes_transferredEv
00000000000f1b40 l    DF .text  0000000000000010              _ZN4asio6detail10service_idINS_22deadline_timer_serviceIN5boost10posix_time5ptimeENS_11time_traitsIS5_EEEEED1Ev
00000000000f27c0 l    DF .text  0000000000000050              _ZN4asio6detail7tss_ptrINS0_10call_stackINS0_15task_io_serviceENS0_27task_io_service_thread_infoEE7contextEED1Ev
000000000031f480 l    DF .text  0000000000000210              asio_detail_posix_thread_function
0000000000316c00 l    DF .text  0000000000000230              _ZN4asio6detail31reactive_socket_connect_op_base10do_performEPNS0_10reactor_opE
00000000000f1ec0 l    DF .text  0000000000000060              _ZN4asio5error6detail14netdb_categoryD1Ev
000000000033d100 l    DF .text  0000000000000510              _ZN4asio6detail23reactive_socket_recv_opINS_17mutable_buffers_1ENS_3ssl6detail5io_opINS_19basic_stream_socketINS_2ip3tcpENS_21stream_socket_serviceIS8_EEEENS4_12handshake_opEN5boost3_bi6bind_tIvNSD_4_mfi3mf1IvN5gcomm13AsioTcpSocketERKSt10error_codeEENSE_5list2INSE_5valueINSD_10shared_ptrISJ_EEEEPFNSD_3argILi1EEEvEEEEEEEE11do_completeEPNS0_15task_io_serviceEPNS0_25task_io_service_operationESM_m
00000000000f1fc0 l    DF .text  0000000000000060              _ZN4asio5error6detail17addrinfo_categoryD1Ev
00000000003122c0 l    DF .text  00000000000009b0              _ZN4asio6detail28reactive_socket_recv_op_baseINS0_17consuming_buffersINS_14mutable_bufferESt5arrayIS3_Lm1EEEEE10do_performEPNS0_10reactor_opE
0000000000323880 l    DF .text  0000000000000bb0              _ZN4asio6detail16service_registry6createINS0_13epoll_reactorEEEPNS_10io_service7serviceERS4_
00000000000f1b00 l    DF .text  0000000000000010              _ZN4asio6detail10service_idINS0_14strand_serviceEED1Ev
0000000000324440 l    DF .text  00000000000004b0              _ZN4asio6detail16service_registry6createINS_22deadline_timer_serviceIN5boost10posix_time5ptimeENS_11time_traitsIS6_EEEEEEPNS_10io_service7serviceERSA_
0000000000357a00 l    DF .text  0000000000000490              _ZN4asio6detail27reactive_socket_recvfrom_opISt5arrayINS_14mutable_bufferELm1EENS_2ip14basic_endpointINS5_3udpEEEN5boost3_bi6bind_tIvNS9_4_mfi3mf2IvN5gcomm13AsioUdpSocketERKSt10error_codemEENSA_5list3INSA_5valueINS9_10shared_ptrISF_EEEEPFNS9_3argILi1EEEvEPFNSP_ILi2EEEvEEEEEE11do_completeEPNS0_15task_io_serviceEPNS0_25task_io_service_operationESI_m
0000000000355fc0 l    DF .text  0000000000000770              _ZN4asio6detail32reactive_socket_recvfrom_op_baseISt5arrayINS_14mutable_bufferELm1EENS_2ip14basic_endpointINS5_3udpEEEE10do_performEPNS0_10reactor_opE
000000000033db40 l    DF .text  0000000000000b90              _ZN4asio6detail23reactive_socket_send_opINS_17mutable_buffers_1ENS0_8write_opINS_19basic_stream_socketINS_2ip3tcpENS_21stream_socket_serviceIS6_EEEES2_NS0_14transfer_all_tENS_3ssl6detail5io_opIS9_NSC_12handshake_opEN5boost3_bi6bind_tIvNSF_4_mfi3mf1IvN5gcomm13AsioTcpSocketERKSt10error_codeEENSG_5list2INSG_5valueINSF_10shared_ptrISL_EEEEPFNSF_3argILi1EEEvEEEEEEEEEE11do_completeEPNS0_15task_io_serviceEPNS0_25task_io_service_operationESO_m
00000000000f1cc0 l    DF .text  0000000000000060              _ZN4asio6detail15system_categoryD1Ev
0000000000322ec0 l    DF .text  00000000000009c0              _ZN4asio6detail16service_registry6createINS_2ip16resolver_serviceINS3_3tcpEEEEEPNS_10io_service7serviceERS7_
0000000000320ac0 l    DF .text  0000000000000de0              _ZN4asio6detail30reactive_socket_accept_op_baseINS_12basic_socketINS_2ip3tcpENS_21stream_socket_serviceIS4_EEEES4_E10do_performEPNS0_10reactor_opE
000000000031f380 l    DF .text  00000000000000d0              _ZN4asio6detail12posix_thread4funcINS0_21resolver_service_base22work_io_service_runnerEE3runEv
0000000000303840 l    DF .text  0000000000000010              _ZN4asio12placeholders5errorEv
0000000000303d00 l    DF .text  0000000000000010              _ZN4asio6detail10service_idINS_21stream_socket_serviceINS_2ip3tcpEEEED1Ev
00000000000f2840 l    DF .text  0000000000000050              _ZN4asio6detail7tss_ptrINS0_10call_stackINS0_14strand_service11strand_implEhE7contextEED1Ev
00000000000f1ac0 l    DF .text  0000000000000010              _ZN4asio6detail10service_idINS0_15task_io_serviceEED1Ev
0000000000316380 l    DF .text  0000000000000300              _ZN4asio6detail18completion_handlerINS0_7binder1IN5boost3_bi6bind_tIvNS3_4_mfi3mf1IvN5gcomm13AsioTcpSocketERKSt10error_codeEENS4_5list2INS4_5valueINS3_10shared_ptrIS9_EEEEPFNS3_3argILi1EEEvEEEEESA_EEE11do_completeEPNS0_15task_io_serviceEPNS0_25task_io_service_operationESC_m
0000000000356bc0 l    DF .text  00000000000009c0              _ZN4asio6detail16service_registry6createINS_2ip16resolver_serviceINS3_3udpEEEEEPNS_10io_service7serviceERS7_
0000000000311b80 l    DF .text  0000000000000710              _ZN4asio6detail28reactive_socket_send_op_baseISt5arrayINS_12const_bufferELm2EEE10do_performEPNS0_10reactor_opE
0000000000064540 l    DF .text  0000000000001610              _GLOBAL__sub_I_asio_protonet.cpp
0000000000062b80 l    DF .text  0000000000001990              _GLOBAL__sub_I_asio_udp.cpp
00000000000612c0 l    DF .text  00000000000018c0              _GLOBAL__sub_I_asio_tcp.cpp
0000000000055400 l    DF .text  0000000000001500              _GLOBAL__sub_I_gu_asio.cpp
00000000000e6040 l    DF .text  00000000000000a0              _ZN4asio3ssl6detail17password_callbackIN5boost3_bi6bind_tINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEENS3_4_mfi4cmf0ISB_N12_GLOBAL__N_119SSLPasswordCallbackEEENS4_5list1INS4_5valueIPSF_EEEEEEE4callEmNS0_12context_base16password_purposeE
scons: *** Error 1
scons: *** [libgalera_smm.so] Error 1
scons: building terminated because of errors.

powerpc

g++ -o gcache/src/GCache_memops.os -c -Wold-style-cast -Weffc++ -std=c++11 -pipe -Wno-long-long -Wno-deprecated -g -O2 -fdebug-prefix-map=/<<PKGBUILDDIR>>=. -fstack-protector-strong -Wformat -Werror=format-security -g -O3 -DNDEBUG -pthread -fPIC -Wall -Wextra -Wno-unused-parameter -fPIC -Wdate-time -D_FORTIFY_SOURCE=2 -D_XOPEN_SOURCE=600 -DHAVE_COMMON_H -DGALERA_USE_GU_NETWORK -DHAVE_BYTESWAP_H -DHAVE_ENDIAN_H -DHAVE_EXECINFO_H -DHAVE_STD_ARRAY -DHAVE_BOOST_SHARED_PTR_HPP -DHAVE_STD_UNORDERED_MAP -DBOOST_DATE_TIME_POSIX_TIME_STD_CONFIG=1 -DHAVE_ASIO_HPP -DOPENSSL_HAS_SET_ECDH_AUTO -Iasio -Iwsrep/src -Icommon -Igalerautils/src gcache/src/GCache_memops.cpp
builder_unit_test(["galerautils/tests/gu_tests++.passed"], ["galerautils/tests/gu_tests++"])
/<<PKGBUILDDIR>>/galerautils/tests/gu_tests++: error while loading shared libraries: R_PPC_REL24 relocation at 0x00b16a7c for symbol `strcmp' out of range
scons: *** [galerautils/tests/gu_tests++.passed] Error 1

sparc64

Running suite(s): DataSet
0%: Checks: 2, Failures: 0, Errors: 2
galera/tests/data_set_check.cpp:243:E:DataSet:ver1:0: (after this point) Received signal 10 (Bus error)
galera/tests/data_set_check.cpp:143:E:DataSet:ver2:0: (after this point) Received signal 10 (Bus error)
Running suite(s): KeySet
0%: Checks: 4, Failures: 0, Errors: 4
galera/tests/key_set_check.cpp:54:E:KeySet:ver1_3:0: (after this point) Received signal 10 (Bus error)
galera/tests/key_set_check.cpp:235:E:KeySet:ver2_3:0: (after this point) Received signal 10 (Bus error)
galera/tests/key_set_check.cpp:235:E:KeySet:ver2_4:0: (after this point) Received signal 10 (Bus error)
galera/tests/key_set_check.cpp:235:E:KeySet:ver2_5:0: (after this point) Received signal 10 (Bus error)
Running suite(s): WriteSet
60%: Checks: 5, Failures: 0, Errors: 2
galera/tests/write_set_ng_check.cpp:83:E:WriteSet basic:ver3_basic_rsv1:0: (after this point) Received signal 10 (Bus error)
galera/tests/write_set_ng_check.cpp:307:E:WriteSet annotation:ver3_annotation_rsv1:0: (after this point) Received signal 10 (Bus error)
Running suite(s): certification
100%: Checks: 5, Failures: 0, Errors: 0
Running suite(s): trx_handle
100%: Checks: 4, Failures: 0, Errors: 0
Running suite(s): service_thd
100%: Checks: 3, Failures: 0, Errors: 0
Running suite(s): ist
100%: Checks: 2, Failures: 0, Errors: 0
Running suite(s): saved_state
100%: Checks: 3, Failures: 0, Errors: 0
Running suite(s): Defaults
100%: Checks: 1, Failures: 0, Errors: 0
Total tests failed: 8
scons: *** [galera/tests/galera_check.passed] Error 1
scons: building terminated because of errors.

Filed downstream at:

ottok commented 4 years ago

It would be nice to see at least a small acknowledgement that somebody read this...

ottok commented 4 years ago

Got these tips from IRC #debian-ports:

18:24 <@cbmuser> on SPARC, the code is broken due to unaligned access: 18:24 <@cbmuser> galera/tests/write_set_ng_check.cpp:83:E:WriteSet basic:ver3_basic_rsv1:0: (after this point) Received signal 10 (Bus error) 18:24 <@cbmuser> galera/tests/write_set_ng_check.cpp:307:E:WriteSet annotation:ver3_annotation_rsv1:0: (after this point) Received signal 10 (Bus error) 18:24 <@cbmuser> should be easy to fix

temeo commented 4 years ago

Fixes to some alignment related issues (fixed for Sparc64) will be pushed in the next Galera release.

ottok commented 3 years ago

Galera 26.4.6 status

Fixed

I confirm that sparc64 is now fixed. Thanks @temeo!

Continue to fail

The following platforms continue to fail for the same reasons as above: ia64, powerpc

hppa is also still failing, but now for a new reason:

Running suite(s): GCS send monitor
80%: Checks: 5, Failures: 1, Errors: 0
gcs/src/unit_tests/gcs_sm_test.cpp:248:F:gcs_sm:gcs_sm_test_pause:0: paused_avg: expected <= 1.000000e-15, got nan

Regressed

The following platforms regressed:

alpha

g++ -o galerautils/tests/deqmap_bench -specs=/usr/share/dpkg/pie-link.specs -Wl,-z,relro -Wl,-z,now galerautils/tests/deqmap_bench.o galerautils/src/libgalerautils++.a galerautils/src/libgalerautils.a -lpthread -latomic -lrt -lssl -lcrypto -lcheck -lm -lsubunit -lrt
collect2: fatal error: ld terminated with signal 11 [Segmentation fault]

hurd-i386

> .sconf_temp/conftest_98d9674fb72071a8ac77baed52538c59_0.c:2:10: fatal error: sys/epoll.h: No such file or directory

Remember that full build overview is available at https://buildd.debian.org/status/package.php?p=galera-4

ottok commented 3 years ago

Galera 26.4.7 status

Unfortunately, quite a lot of regressions:

https://buildd.debian.org/status/package.php?p=galera-4

temeo commented 3 years ago

armel, mipsel, m68k, powerpc, sh4, : /obj-arm-linux-gnueabi/galerautils/tests/./galerautils/tests/gu_atomic_test.cpp:86: undefined reference to__atomic_fetch_sub_8'`

alpha, hppa, hurd-i386, : /gcs/src/unit_tests/gcs_sm_test.cpp:248:F:gcs_sm:gcs_sm_test_pause:0: paused_avg: expected <= 1.000000e-15, got nan

ia64:

Checking library symbol visibility (hidden)
cd /<<PKGBUILDDIR>>/obj-ia64-linux-gnu/galera/src && sh -c "! objdump -T /<<PKGBUILDDIR>>/obj-ia64-linux-gnu/libgalera_smm.so | grep asio 1> /dev/null"
make[3]: *** [galera/src/CMakeFiles/galera_smm.dir/build.make:118: libgalera_smm.so] Error 1
make[3]: *** Deleting file 'libgalera_smm.so'

sparc64:

50%: Checks: 2, Failures: 0, Errors: 1
./galera/tests/data_set_check.cpp:216:E:DataSet:ver1:0: (after this point) Received signal 10 (Bus error)
Running suite(s): KeySet
75%: Checks: 4, Failures: 0, Errors: 1
./galera/tests/key_set_check.cpp:93:E:KeySet:ver1_3:0: (after this point) Received signal 10 (Bus error)
Running suite(s): WriteSet
80%: Checks: 5, Failures: 0, Errors: 1
./galera/tests/write_set_ng_check.cpp:101:E:WriteSet basic:ver3_basic_rsv1:0: (after this point) Received signal 10 (Bus error)

temeo commented 3 years ago

armel, mipsel, m68k, powerpc, sh4, : /obj-arm-linux-gnueabi/galerautils/tests/./galerautils/tests/gu_atomic_test.cpp:86: undefined reference to__atomic_fetch_sub_8'`

For these one possible workaround is to export DEB_LDFLAGS_MAINT_APPEND=-latomic for affected platforms, as in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=909938

Proper fix should probably go into cmake scripts.

ottok commented 3 years ago

A Debian dev actually submitted a patch for cross-build fixes (that regressed in latest galera-4), maybe they help for the other platforms as well? See https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=981652 and the patch attached to it.

temeo commented 3 years ago

The patch in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=981652 looks good (will include in the next release), though it does not resolve any of the problems above.

Could you check if the following cmake patch solves the atomic problem:


diff --git a/CMakeLists.txt b/CMakeLists.txt
index 73897b80..652117fe 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -11,6 +11,7 @@ include(CheckCXXCompilerFlag)
 include(CheckIncludeFile)
 include(CheckIncludeFileCXX)
 include(CheckCXXCompilerFlag)
+include(CheckLibraryExists)

 include_directories(
   ${CMAKE_SOURCE_DIR}
diff --git a/cmake/os.cmake b/cmake/os.cmake
index 4ef84bdb..274b046f 100644
--- a/cmake/os.cmake
+++ b/cmake/os.cmake
@@ -6,6 +6,13 @@

 find_library(PTHREAD_LIB pthread)
 find_library(RT_LIB rt)
+find_library(ATOMIC_LIB atomic)
 set(GALERA_SYSTEM_LIBS ${PTHREAD_LIB} ${RT_LIB})

+check_library_exists(atomic _atomic_fetch_add_8 "" GALERA_HAVE_ATOMIC_LIB)
+if (GALERA_HAVE_ATOMIC_LIB)
+  find_library(ATOMIC_LIB atomic)
+  set(GALERA_SYSTEM_LIBS "${GALERA_SYSTEM_LIBS} ${ATOMIC_LIB}")
+endif()
+
 message(STATUS "Galera system libs: ${GALERA_SYSTEM_LIBS}")
``

temeo commented 3 years ago

The above cmake patch has invalid syntax, this should be better:

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 73897b80..652117fe 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -11,6 +11,7 @@ include(CheckCXXCompilerFlag)
 include(CheckIncludeFile)
 include(CheckIncludeFileCXX)
 include(CheckCXXCompilerFlag)
+include(CheckLibraryExists)

 include_directories(
   ${CMAKE_SOURCE_DIR}
diff --git a/cmake/os.cmake b/cmake/os.cmake
index 4ef84bdb..d21217b9 100644
--- a/cmake/os.cmake
+++ b/cmake/os.cmake
@@ -8,4 +8,10 @@ find_library(PTHREAD_LIB pthread)
 find_library(RT_LIB rt)
 set(GALERA_SYSTEM_LIBS ${PTHREAD_LIB} ${RT_LIB})

+check_library_exists(atomic _atomic_fetch_add_8 "" GALERA_HAVE_ATOMIC_LIB)
+if (GALERA_HAVE_ATOMIC_LIB)
+  find_library(ATOMIC_LIB atomic)
+  list(APPEND GALERA_SYSTEM_LIBS ${ATOMIC_LIB})
+endif()
+
 message(STATUS "Galera system libs: ${GALERA_SYSTEM_LIBS}")

ottok commented 3 years ago

Testing on Salsa-CI first at https://salsa.debian.org/mariadb-team/galera-4/-/pipelines/227026 and will later proceed to upload a new galera-4 to Debian.

ottok commented 3 years ago

See https://buildd.debian.org/status/package.php?p=galera-4 for latest status Armel is now failing on another issue.

temeo commented 3 years ago

It seems to still fail also on atomic issue. I need to find Armel machine to troubleshoot further.

ottok commented 3 years ago

Note that both armel and mipsel worked in 26.4.6 with scons. This is probably a cmake implementation regression.

temeo commented 3 years ago

So, there was a typo in above patch (missing underscore). Also find_library(ATOMIC_LIB atomic) failed on mipsel because linker could not find libatomic.so in default configuration. It finds libatomic.so.1 though.

Revised the patch and tested compilation on mipsel and x86_64:

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 73897b80..652117fe 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -11,6 +11,7 @@ include(CheckCXXCompilerFlag)
 include(CheckIncludeFile)
 include(CheckIncludeFileCXX)
 include(CheckCXXCompilerFlag)
+include(CheckLibraryExists)

 include_directories(
   ${CMAKE_SOURCE_DIR}
diff --git a/cmake/os.cmake b/cmake/os.cmake
index 4ef84bdb..431c2048 100644
--- a/cmake/os.cmake
+++ b/cmake/os.cmake
@@ -8,4 +8,18 @@ find_library(PTHREAD_LIB pthread)
 find_library(RT_LIB rt)
 set(GALERA_SYSTEM_LIBS ${PTHREAD_LIB} ${RT_LIB})

+# Check if linkage with atomic library is needed for 8 byte atomics
+set(ATOMIC_8_TEST_C_SOURCE
+  "int main() { long long val; __atomic_fetch_add_8(&val, 1, __ATOMIC_SEQ_CST); return 0;}")
+check_c_source_compiles("${ATOMIC_8_TEST_C_SOURCE}" GALERA_HAVE_ATOMIC)
+if (NOT GALERA_HAVE_ATOMIC)
+  find_library(ATOMIC_LIB NAMES atomic libatomic.so.1)
+  set(CMAKE_REQUIRED_LIBRARIES ${ATOMIC_LIB})
+  check_c_source_compiles("${ATOMIC_8_TEST_C_SOURCE}" GALERA_HAVE_ATOMIC_LIB)
+  if (NOT GALERA_HAVE_ATOMIC_LIB)
+    message(FATAL_ERROR "Could not find support for 64 bit atomic operations")
+  endif()
+  unset(CMAKE_REQUIRED_LIBRARIES)
+  list(APPEND GALERA_SYSTEM_LIBS ${ATOMIC_LIB})
+endif()
 message(STATUS "Galera system libs: ${GALERA_SYSTEM_LIBS}")

ottok commented 3 years ago

Thanks, seems to work now! I uploaded to Debian experimental and all official archs are now green: https://buildd.debian.org/status/package.php?p=galera-4&suite=experimental

ottok commented 3 years ago

In the latest build of Galera 25.3.33 hppa and sparc64 regressed:

ottok commented 3 years ago

I uploaded Galera 26.4.8 now to Debian, and multiple builds fail on the test step, apparently due to timeouts.

Do you think this should work if I reintroduce the CK_TIMEOUT_MULTIPLIER=2 on CMake? We hade it with SCons.

diff --git a/debian/rules b/debian/rules
index aa3981e..cc9a9e6 100755
--- a/debian/rules
+++ b/debian/rules
@@ -7,6 +7,12 @@ DPKG_EXPORT_BUILDFLAGS = 1
 # Include all defaults, including buildflags.mk
 include /usr/share/dpkg/default.mk

+# Some mipsel buildd's are a bit slow and fail tests if they are not
+# given enough time
+ifeq (mipsel,$(DEB_HOST_ARCH))
+  export CK_TIMEOUT_MULTIPLIER=2
+endif
+

temeo commented 3 years ago

Yes, that multiplier is needed on some platforms. Dropping it from rules was not intentional.

The multiplier could be even higher, say 5, and maybe it should be set on all archs just in case.

ottok commented 2 years ago

After uploading 26.4.10-1 to Debian I saw this test failure:

Running tests...
/usr/bin/ctest --force-new-ctest-process --output-on-failure
Test project /<<PKGBUILDDIR>>/obj-x86_64-linux-gnu
    Start 1: gu_tests
1/7 Test #1: gu_tests .........................   Passed    3.35 sec
    Start 2: gu_tests++
2/7 Test #2: gu_tests++ .......................***Failed    1.73 sec
Running suite(s): gu::Atomic
100%: Checks: 3, Failures: 0, Errors: 0
Running suite(s): gu::Vector
100%: Checks: 2, Failures: 0, Errors: 0
Running suite(s): gu::String
100%: Checks: 2, Failures: 0, Errors: 0
Running suite(s): gu::vlq
100%: Checks: 4, Failures: 0, Errors: 0
Running suite(s): gu::Hash
100%: Checks: 2, Failures: 0, Errors: 0
Running suite(s): gu::MemPool
100%: Checks: 2, Failures: 0, Errors: 0
Running suite(s): gu::Allocator
100%: Checks: 1, Failures: 0, Errors: 0
Running suite(s): gu::RecordSet
100%: Checks: 6, Failures: 0, Errors: 0
Running suite(s): String Utils
100%: Checks: 3, Failures: 0, Errors: 0
Running suite(s): galerautils++ URI
100%: Checks: 6, Failures: 0, Errors: 0
Running suite(s): gu::GTID
100%: Checks: 1, Failures: 0, Errors: 0
Running suite(s): gu::Config
100%: Checks: 1, Failures: 0, Errors: 0
Running suite(s): galerautils++ Networking
100%: Checks: 1, Failures: 0, Errors: 0
Running suite(s): gu::datetime
100%: Checks: 4, Failures: 0, Errors: 0
Running suite(s): gu::Histogram
100%: Checks: 1, Failures: 0, Errors: 0
Running suite(s): gu::Stats
100%: Checks: 1, Failures: 0, Errors: 0
Running suite(s): galerautils Thread
100%: Checks: 2, Failures: 0, Errors: 0
Running suite(s): gu::asio
terminate called after throwing an instance of 'gu::Exception'
  what():  error opening datagram socketudp://127.0.0.1:0: 1 (Operation not permitted)
     at ./galerautils/src/gu_asio_datagram.cpp:resolve_and_open():107
terminate called after throwing an instance of 'gu::Exception'
  what():  error opening datagram socketudp://127.0.0.1:0: 1 (Operation not permitted)
     at ./galerautils/src/gu_asio_datagram.cpp:resolve_and_open():107
terminate called after throwing an instance of 'gu::Exception'
  what():  error opening datagram socketudp://127.0.0.1:0: 1 (Operation not permitted)
     at ./galerautils/src/gu_asio_datagram.cpp:resolve_and_open():107
terminate called after throwing an instance of 'gu::Exception'
  what():  error opening datagram socketudp://127.0.0.1:0: 1 (Operation not permitted)
     at ./galerautils/src/gu_asio_datagram.cpp:resolve_and_open():107
94%: Checks: 77, Failures: 0, Errors: 4
./galerautils/tests/gu_asio_test.cpp:1811:E:test_datagram_open:test_datagram_open:0: (after this point) Received signal 6 (Aborted)
./galerautils/tests/gu_asio_test.cpp:1820:E:test_datagram_connect:test_datagram_connect:0: (after this point) Received signal 6 (Aborted)
./galerautils/tests/gu_asio_test.cpp:1829:E:test_datagram_open_connect:test_datagram_open_connect:0: (after this point) Received signal 6 (Aborted)
./galerautils/tests/gu_asio_test.cpp:1894:E:test_datagram_send_to_and_async_read:test_datagram_send_to_and_async_read:0: (after this point) Received signal 6 (Aborted)
Running suite(s): gu::DeqMap
100%: Checks: 11, Failures: 0, Errors: 0
Total tests failed: 4

    Start 3: check_gcomm
3/7 Test #3: check_gcomm ......................   Passed    0.99 sec
    Start 4: gcache_tests
4/7 Test #4: gcache_tests .....................   Passed    0.02 sec
    Start 5: gcs_tests
5/7 Test #5: gcs_tests ........................   Passed    9.00 sec
    Start 6: galera_check
6/7 Test #6: galera_check .....................   Passed    1.88 sec
    Start 7: wsrep_test
7/7 Test #7: wsrep_test .......................   Passed    0.01 sec

86% tests passed, 1 tests failed out of 7

Total Test time (real) =  16.99 sec

The following tests FAILED:
      2 - gu_tests++ (Failed)
Errors while running CTest

The issue is sporadic, as it went away on a rebuild. Full log at https://buildd.debian.org/status/fetch.php?pkg=galera-4&arch=amd64&ver=26.4.10-1&stamp=1646201146&raw=0

ottok commented 2 years ago

For reference, build status for latest upload to Debian:

Looking good!

ottok commented 1 year ago

With latest Galera 26.4.14 I saw several post-build test failures that all are different, but also seemed sporadic.

arch i386 build of 26.4.14-1 on Salsa-CI

The i386 build sporadically fails on Salsa-CI on disk space issue:

Test project /builds/mariadb-team/galera-4/debian/output/source_dir/obj-i686-linux-gnu
    Start 1: gu_tests
1/7 Test #1: gu_tests .........................   Passed    3.70 sec
    Start 2: gu_tests++
2/7 Test #2: gu_tests++ .......................   Passed    1.45 sec
    Start 3: check_gcomm
3/7 Test #3: check_gcomm ......................   Passed    0.57 sec
    Start 4: gcache_tests
4/7 Test #4: gcache_tests .....................   Passed    0.01 sec
    Start 5: gcs_tests
5/7 Test #5: gcs_tests ........................   Passed    8.94 sec
    Start 6: galera_check
6/7 Test #6: galera_check .....................***Failed    2.83 sec
Running suite(s): DataSet
100%: Checks: 2, Failures: 0, Errors: 0
Running suite(s): KeySet
100%: Checks: 4, Failures: 0, Errors: 0
Running suite(s): WriteSet
100%: Checks: 5, Failures: 0, Errors: 0
Running suite(s): certification
100%: Checks: 6, Failures: 0, Errors: 0
Running suite(s): trx_handle
100%: Checks: 4, Failures: 0, Errors: 0
Running suite(s): service_thd
100%: Checks: 3, Failures: 0, Errors: 0
Running suite(s): ist
100%: Checks: 14, Failures: 0, Errors: 0
Running suite(s): saved_state
100%: Checks: 3, Failures: 0, Errors: 0
Running suite(s): Defaults
Requested size 134219032 for './galera.cache' exceeds available storage space 94769152: 28 (No space left on device)
     at ./galerautils/src/gu_fdesc.cpp:FileDescriptor():104
0%: Checks: 1, Failures: 1, Errors: 0
./galera/tests/defaults_check.cpp:276:F:defaults:defaults:0: Assertion 'WSREP_OK == ret' failed
Running suite(s): progress_suite
100%: Checks: 1, Failures: 0, Errors: 0
Total tests failed: 1
    Start 7: wsrep_test
7/7 Test #7: wsrep_test .......................   Passed    0.00 sec
86% tests passed, 1 tests failed out of 7

Log example: https://salsa.debian.org/mariadb-team/galera-4/-/jobs/4140942

Does is really need 100 MB extra space to run the test, or is that an anomaly?

arch amd64 build of 26.4.14-1 Debian buildd

The arch amd64 build for 26.4.13-1 passed. Now it had a sporadic failure on:

Test project /<<PKGBUILDDIR>>/obj-x86_64-linux-gnu
    Start 1: gu_tests
1/7 Test #1: gu_tests .........................   Passed    3.17 sec
    Start 2: gu_tests++
2/7 Test #2: gu_tests++ .......................***Failed    2.11 sec
Running suite(s): gu::Atomic
100%: Checks: 3, Failures: 0, Errors: 0
Running suite(s): gu::Vector
100%: Checks: 2, Failures: 0, Errors: 0
Running suite(s): gu::String
100%: Checks: 2, Failures: 0, Errors: 0
Running suite(s): gu::vlq
100%: Checks: 4, Failures: 0, Errors: 0
Running suite(s): gu::Hash
100%: Checks: 2, Failures: 0, Errors: 0
Running suite(s): gu::MemPool
100%: Checks: 2, Failures: 0, Errors: 0
Running suite(s): gu::Allocator
100%: Checks: 1, Failures: 0, Errors: 0
Running suite(s): gu::RecordSet
100%: Checks: 6, Failures: 0, Errors: 0
Running suite(s): String Utils
100%: Checks: 3, Failures: 0, Errors: 0
Running suite(s): galerautils++ URI
100%: Checks: 6, Failures: 0, Errors: 0
Running suite(s): gu::GTID
100%: Checks: 1, Failures: 0, Errors: 0
Running suite(s): gu::Config
100%: Checks: 1, Failures: 0, Errors: 0
Running suite(s): galerautils++ Networking
100%: Checks: 1, Failures: 0, Errors: 0
Running suite(s): gu::datetime
100%: Checks: 7, Failures: 0, Errors: 0
Running suite(s): gu::Histogram
100%: Checks: 1, Failures: 0, Errors: 0
Running suite(s): gu::Stats
100%: Checks: 1, Failures: 0, Errors: 0
Running suite(s): galerautils Thread
100%: Checks: 2, Failures: 0, Errors: 0
Running suite(s): gu::asio
terminate called after throwing an instance of 'gu::Exception'
  what():  error opening datagram socketudp://127.0.0.1:0: 1 (Operation not permitted)
     at ./galerautils/src/gu_asio_datagram.cpp:resolve_and_open():107
terminate called after throwing an instance of 'gu::Exception'
  what():  error opening datagram socketudp://127.0.0.1:0: 1 (Operation not permitted)
     at ./galerautils/src/gu_asio_datagram.cpp:resolve_and_open():107
terminate called after throwing an instance of 'gu::Exception'
  what():  error opening datagram socketudp://127.0.0.1:0: 1 (Operation not permitted)
     at ./galerautils/src/gu_asio_datagram.cpp:resolve_and_open():107
terminate called after throwing an instance of 'gu::Exception'
  what():  error opening datagram socketudp://127.0.0.1:0: 1 (Operation not permitted)
     at ./galerautils/src/gu_asio_datagram.cpp:resolve_and_open():107
94%: Checks: 77, Failures: 0, Errors: 4
./galerautils/tests/gu_asio_test.cpp:1864:E:test_datagram_open:test_datagram_open:0: (after this point) Received signal 6 (Aborted)
./galerautils/tests/gu_asio_test.cpp:1873:E:test_datagram_connect:test_datagram_connect:0: (after this point) Received signal 6 (Aborted)
./galerautils/tests/gu_asio_test.cpp:1882:E:test_datagram_open_connect:test_datagram_open_connect:0: (after this point) Received signal 6 (Aborted)
./galerautils/tests/gu_asio_test.cpp:1947:E:test_datagram_send_to_and_async_read:test_datagram_send_to_and_async_read:0: (after this point) Received signal 6 (Aborted)
Running suite(s): gu::DeqMap
100%: Checks: 11, Failures: 0, Errors: 0
Running suite(s): gu::utils
100%: Checks: 2, Failures: 0, Errors: 0
Total tests failed: 4

    Start 3: check_gcomm
3/7 Test #3: check_gcomm ......................   Passed    1.00 sec
    Start 4: gcache_tests
4/7 Test #4: gcache_tests .....................   Passed    0.02 sec
    Start 5: gcs_tests
5/7 Test #5: gcs_tests ........................   Passed    9.00 sec
    Start 6: galera_check
6/7 Test #6: galera_check .....................   Passed    3.15 sec
    Start 7: wsrep_test
7/7 Test #7: wsrep_test .......................   Passed    0.01 sec

86% tests passed, 1 tests failed out of 7

Full log: https://buildd.debian.org/status/fetch.php?pkg=galera-4&arch=amd64&ver=26.4.14-1%7Eexp1&stamp=1681597206&raw=0

However, it passed on rebuild.

arch hppa build of 26.4.14-1 on Debian buildd

The arch hppa build for 26.4.13-1 passed. Now it no longer passes on:

Test project /<<PKGBUILDDIR>>/obj-hppa-linux-gnu
    Start 1: gu_tests
1/7 Test #1: gu_tests .........................***Failed    3.96 sec
Running suite(s): Galera memory utils
100%: Checks: 1, Failures: 0, Errors: 0
Running suite(s): Galera byteswap functions
100%: Checks: 1, Failures: 0, Errors: 0
Running suite(s): FNV hash
100%: Checks: 3, Failures: 0, Errors: 0
Running suite(s): MurmurHash3
100%: Checks: 3, Failures: 0, Errors: 0
Running suite(s): Spooky hash
100%: Checks: 1, Failures: 0, Errors: 0
Running suite(s): CRC32C implementations
100%: Checks: 3, Failures: 0, Errors: 0
Running suite(s): Galera hash
100%: Checks: 3, Failures: 0, Errors: 0
Running suite(s): Galera dbug functions
0%: Checks: 1, Failures: 0, Errors: 1
./galerautils/tests/gu_dbug_test.c:48:E:gu_dbug:gu_dbug_test:0: (after this point) Received signal 11 (Segmentation fault)
Running suite(s): Galera time functions
100%: Checks: 1, Failures: 0, Errors: 0
Running suite(s): Galera FIFO functions
100%: Checks: 2, Failures: 0, Errors: 0
Running suite(s): Galera UUID utils
100%: Checks: 1, Failures: 0, Errors: 0
Running suite(s): Galera LOCK_STEP utils
100%: Checks: 2, Failures: 0, Errors: 0
Running suite(s): Galera Str util suite
100%: Checks: 3, Failures: 0, Errors: 0
Running suite(s): Galera misc utils functions
100%: Checks: 1, Failures: 0, Errors: 0
Total tests failed: 1

    Start 2: gu_tests++
2/7 Test #2: gu_tests++ .......................   Passed   37.69 sec
    Start 3: check_gcomm
3/7 Test #3: check_gcomm ......................   Passed   22.76 sec
    Start 4: gcache_tests
4/7 Test #4: gcache_tests .....................   Passed    0.52 sec
    Start 5: gcs_tests
5/7 Test #5: gcs_tests ........................   Passed   11.13 sec
    Start 6: galera_check
6/7 Test #6: galera_check .....................   Passed    9.16 sec
    Start 7: wsrep_test
7/7 Test #7: wsrep_test .......................   Passed    0.12 sec

86% tests passed, 1 tests failed out of 7

Full log at https://buildd.debian.org/status/fetch.php?pkg=galera-4&arch=hppa&ver=26.4.14-1%7Eexp1&stamp=1681610582&raw=0

Full overview of Galera builds in Debian experimental also shows arch alpha and sparc64 failing on post-build tests, but those also failed on 26.4.13.

arch s390x build of 26.4.14-1 on Ubuntu Launchpad

The arch s390x build for 26.4.13-1 passed on Launchpad. Now it is failing on:

Test project /<<PKGBUILDDIR>>/obj-s390x-linux-gnu
    Start 1: gu_tests
1/7 Test #1: gu_tests .........................   Passed    3.04 sec
    Start 2: gu_tests++
2/7 Test #2: gu_tests++ .......................   Passed    1.12 sec
    Start 3: check_gcomm
3/7 Test #3: check_gcomm ......................   Passed    0.65 sec
    Start 4: gcache_tests
4/7 Test #4: gcache_tests .....................***Failed    0.01 sec
Running suite(s): gcache::MemStore
100%: Checks: 1, Failures: 0, Errors: 0
Running suite(s): gcache::RbStore
50%: Checks: 2, Failures: 1, Errors: 0
/<<PKGBUILDDIR>>/gcache/tests/gcache_rb_test.cpp:349:F:recovery:recovery:0: Assertion '!ctx.s2p.empty()' failed
Running suite(s): gcache::PageStore
100%: Checks: 3, Failures: 0, Errors: 0
Total tests failed: 1

    Start 5: gcs_tests
5/7 Test #5: gcs_tests ........................   Passed    8.94 sec
    Start 6: galera_check
6/7 Test #6: galera_check .....................   Passed    2.87 sec
    Start 7: wsrep_test
7/7 Test #7: wsrep_test .......................   Passed    0.00 sec

86% tests passed, 1 tests failed out of 7

Full log at https://launchpadlibrarian.net/661392360/buildlog_ubuntu-focal-s390x.galera-4_26.4.14-0+deb11u1~ubuntu20.04.1~1681590767.0f6cb44+bullseye.next_BUILDING.txt.gz

ottok commented 1 year ago

For reference, latest build status in Debian:

I would need help in particular with https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=970043 that is failing on:

[ 96%] Linking CXX static library libgalera_smm_static.a
cd /<<PKGBUILDDIR>>/obj-ia64-linux-gnu/galera/src && /usr/bin/cmake -P CMakeFiles/galera_smm_static.dir/cmake_clean_target.cmake
cd /<<PKGBUILDDIR>>/obj-ia64-linux-gnu/galera/src && /usr/bin/cmake -E cmake_link_script CMakeFiles/galera_smm_static.dir/link.txt --verbose=1
/usr/bin/ar qc libgalera_smm_static.a CMakeFiles/galera_smm_static.dir/wsrep_provider.cpp.o
/usr/bin/ranlib libgalera_smm_static.a
Checking library symbol visibility (hidden)
sh -c "! /usr/bin/objdump -T libgalera_smm.so | grep asio 1> /dev/null"
make[3]: *** [galera/src/CMakeFiles/galera_smm.dir/build.make:113: libgalera_smm.so] Error 1
make[3]: *** Deleting file 'libgalera_smm.so'
make[3]: Leaving directory '/<<PKGBUILDDIR>>/obj-ia64-linux-gnu'
make[2]: *** [CMakeFiles/Makefile2:1840: galera/src/CMakeFiles/galera_smm.dir/all] Error 2
make[2]: *** Waiting for unfinished jobs....
make[3]: Leaving directory '/<<PKGBUILDDIR>>/obj-ia64-linux-gnu'
[ 96%] Built target galera_smm_static
make[2]: Leaving directory '/<<PKGBUILDDIR>>/obj-ia64-linux-gnu'
make[1]: *** [Makefile:169: all] Error 2
make[1]: Leaving directory '/<<PKGBUILDDIR>>/obj-ia64-linux-gnu'
dh_auto_build: error: cd obj-ia64-linux-gnu && make -j2 "INSTALL=install --strip-program=true" VERBOSE=1 returned exit code 2
make: *** [debian/rules:25: binary-arch] Error 25
dpkg-buildpackage: error: debian/rules binary-arch subprocess returned exit status 2

temeo commented 1 year ago

Hi @ottok

I checked the build logs, and unfortunately there is no indication about what goes wrong. My best guess is that the objdump check for hidden/not hidden symbols crash/fail, but the stdout is redirected to /dev/null so it is not certain.

Would it be possible to apply the following patch to see the full output of objdump command?

diff --git a/galera/src/CMakeLists.txt b/galera/src/CMakeLists.txt
index bea0d40a..350afcfd 100644
--- a/galera/src/CMakeLists.txt
+++ b/galera/src/CMakeLists.txt
@@ -129,7 +129,7 @@ endif()
 if (GALERA_VERSION_SCRIPT)
   add_custom_command(TARGET galera_smm POST_BUILD
     COMMAND
-    sh -c "! ${CMAKE_OBJDUMP} -T libgalera_smm.so | grep asio 1> /dev/null"
+    sh -c "! ${CMAKE_OBJDUMP} -T libgalera_smm.so | grep asio"
     WORKING_DIRECTORY "${PROJECT_BINARY_DIR}"
     COMMENT "Checking library symbol visibility (hidden)"
     VERBATIM)
@@ -137,7 +137,7 @@ else()
   set(GALERA_LINK_OPTIONS "")
   add_custom_command(TARGET galera_smm POST_BUILD
     COMMAND
-    sh -c "${CMAKE_OBJDUMP} -T libgalera_smm.so | grep asio 1> /dev/null"
+    sh -c "${CMAKE_OBJDUMP} -T libgalera_smm.so | grep asio"
     WORKING_DIRECTORY "${PROJECT_BINARY_DIR}"
     COMMENT "Checking library symbol visibility (not hidden)"
     VERBATIM)
@@ -147,7 +147,7 @@ if (NOT GALERA_WITH_SSL)
   message(STATUS "Building Galera without SSL")
   add_custom_command(TARGET galera_smm POST_BUILD
     COMMAND
-    sh -c "! (${CMAKE_OBJDUMP} -x libgalera_smm.so | grep NEEDED.*crypto 1> /dev/null) && ! (${CMAKE_OBJDUMP} -x libgalera_smm.so | grep NEEDED.*ssl 1> /dev/null)"
+    sh -c "! (${CMAKE_OBJDUMP} -x libgalera_smm.so | grep NEEDED.*crypto) && ! (${CMAKE_OBJDUMP} -x libgalera_smm.so | grep NEEDED.*ssl)"
     WORKING_DIRECTORY "${PROJECT_BINARY_DIR}"
     COMMENT "Verifying that library is not linked with SSL"
     VERBATIM)
@@ -156,7 +156,7 @@ else()
     message(STATUS "Building Galera with static SSL")
     add_custom_command(TARGET galera_smm POST_BUILD
       COMMAND
-      sh -c "(${CMAKE_OBJDUMP} -t libgalera_smm.so | grep OPENSSL 1> /dev/null) && (${CMAKE_OBJDUMP} -t libgalera_smm.so | grep CRYPTO 1> /dev/null)"
+      sh -c "(${CMAKE_OBJDUMP} -t libgalera_smm.so | grep OPENSSL 1) && (${CMAKE_OBJDUMP} -t libgalera_smm.so | grep CRYPTO)"
       WORKING_DIRECTORY "${PROJECT_BINARY_DIR}"
       COMMENT "Verifying that library has OpenSSL linked statically"
       VERBATIM)
@@ -164,7 +164,7 @@ else()
     message(STATUS "Building Galera with SSL")
     add_custom_command(TARGET galera_smm POST_BUILD
       COMMAND
-      sh -c "(${CMAKE_OBJDUMP} -x libgalera_smm.so | grep NEEDED.*crypto 1> /dev/null) && (${CMAKE_OBJDUMP} -x libgalera_smm.so | grep NEEDED.*ssl 1> /dev/null)"
+      sh -c "(${CMAKE_OBJDUMP} -x libgalera_smm.so | grep NEEDED.*crypto) && (${CMAKE_OBJDUMP} -x libgalera_smm.so | grep NEEDED.*ssl)"
       COMMENT "Verifying that library is linked with SSL"
       WORKING_DIRECTORY "${PROJECT_BINARY_DIR}"
       VERBATIM)

ottok commented 1 year ago

Yes, I can apply that patch and re-upload. Is there anything else I could debug in the same upload?

The amd64 builds failed twice before they passed, both on the post-build test suite failing on permissions and being dying on signal 6:

Any idea what is going on? Can the post-build test be made more stable?

The sparc64 build also fails on one single post-build test:

https://buildd.debian.org/status/fetch.php?pkg=galera-4&arch=sparc64&ver=26.4.16-1&stamp=1695521394&raw=0 (bug report: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1053183)

Also related to post-build tests: @grooverdan submitted the patch https://salsa.debian.org/mariadb-team/galera-4/-/commit/100a53775fa96d664e7d9b43819be9460b3bdc93 for Debian - Could this be adopted upstream?

temeo commented 1 year ago

The amd64 builds failed twice before they passed, both on the post-build test suite failing on permissions and being dying on signal 6:

https://buildd.debian.org/status/fetch.php?pkg=galera-4&arch=amd64&ver=26.4.16-1&stamp=1694977012&raw=0

https://buildd.debian.org/status/fetch.php?pkg=galera-4&arch=amd64&ver=26.4.16-1&stamp=1694933123&raw=0

Any idea what is going on? Can the post-build test be made more stable?

The reason was probably that the tests were not able to open UDP socket on localhost, which caused abort due to unhandled exceptions.

This patch disables the tests which use UDP when it is not available:

diff --git a/galerautils/tests/gu_asio_test.cpp b/galerautils/tests/gu_asio_test.cpp
index c4c948bd..aec4346d 100644
--- a/galerautils/tests/gu_asio_test.cpp
+++ b/galerautils/tests/gu_asio_test.cpp
@@ -1834,6 +1834,20 @@ END_TEST
 // Datagram
 //

+/* Helper to determine if UDP sockets can be opened. */
+static bool have_datagram() try
+{
+  gu::AsioIoService io_service;
+  gu::URI uri("udp://127.0.0.1:0");
+  auto socket(io_service.make_datagram_socket(uri));
+  socket->open(uri);
+  return false;
+}
+catch (...)
+{
+  return false;
+}
+
 class MockDatagramSocketHandler : public gu::AsioDatagramSocketHandler
 {
 public:
@@ -2339,6 +2353,7 @@ Suite* gu_asio_suite()
     //
     // Datagram
     //
+    if (have_datagram()) {

     tc = tcase_create("test_datagram_socket");
     tcase_add_test(tc, test_datagram_socket);
@@ -2360,6 +2375,7 @@ Suite* gu_asio_suite()
     tcase_add_test(tc, test_datagram_send_to_and_async_read);
     suite_add_tcase(s, tc);

+    }
 #if defined(GALERA_ASIO_TEST_MULTICAST)
     tc = tcase_create("test_datagram_connect_multicast");
     tcase_add_test(tc, test_datagram_connect_multicast);

ayurchen commented 1 year ago

About the patch: no reason to change gcache.page_size, gcache.size could be set to 1M. But I'm not sure it is the right thing to do...

grooverdan commented 1 year ago

Is there something untested by setting the test size to so small?

ottok commented 1 year ago

Otto: The sparc64 build also fails on one single post-build test:

https://buildd.debian.org/status/fetch.php?pkg=galera-4&arch=sparc64&ver=26.4.16-1&stamp=1695521394&raw=0 (bug report: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1053183)

@temeo What is best way to skip the test that sparc64 builds are failing on?

ottok commented 1 year ago

Additionally, due to some unidentified change in Debian or Salsa CI runners, the i386 builds have started failing on disk space:

Running suite(s): gu::RecordSet
terminate called after throwing an instance of 'gu::Exception'
  what():  Requested size 67108864 for 'gu_rset_test_ver2.000000' exceeds available storage space 42741760: 28 (No space left on device)
     at ./galerautils/src/gu_fdesc.cpp:FileDescriptor():104: 12 (Cannot allocate memory)
     at ./galerautils/src/gu_alloc.cpp:my_new_page():83
83%: Checks: 6, Failures: 0, Errors: 1
./galerautils/tests/gu_rset_test.cpp:256:E:RecordSet v2:ver2:0: (after this point) Received signal 6 (Aborted)

Can you advice me on how I could skip that test selectively? Maybe I could set some environment variable that CI runner is low on disk space and that test should be skipped? Or is there some allowlist type of mechanism that I can tell the test harness that specific tests are allowed to fail?

ottok commented 1 year ago

The reason was probably that the tests were not able to open UDP socket on localhost, which caused abort due to unhandled exceptions.

This patch disables the tests which use UDP when it is not available: ...

I found this bug report that suggests that root cause is actually lack of IPv6 support: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1007954 - Does that make sense to you?

ottok commented 1 year ago

@temeo What is best way to skip the test that sparc64 builds are failing on?

ayurchen commented 1 year ago

@grooverdan

Is there something untested by setting the test size to so small?

well, the whole point of this test is to test default values. so the only thing that will be untested is the default value of gcache.size )

temeo commented 12 months ago

Hi @ottok

We have made a bunch of fixes which should address the issues discussed above, and also https://github.com/codership/galera/issues/647. I cherry-picked the relevant ones on the top of release_26.4.16 and pushed into https://github.com/codership/galera/tree/4.x-26.4.16-gh-558. These fixes will be included in the next release, but I don't know when it will be available.

We used UBSAN to track down the cases with unaligned memory access, which is the most likely reason for sparc64 test failures. See below comments on individual commits in that branch:

CMake patch to enable UBSAN: https://github.com/codership/galera/commit/cb4437f7f4b30ff1d3394fa10fa1e4168b47b61c.

Fix for atomics check to make UBSAN build succeed with clang: https://github.com/codership/galera/commit/e0431ecfde2af1e361c133a0e47143c255282d42.

Fix for failures in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1053183: https://github.com/codership/galera/commit/56c6e4a10bd179eb306e9a7cc4abee8f1f309d42. The failing tests were for old write set formats which are required for backwards compatibility when upgrading from Galera 3, but are not used otherwise in Galera 4. Those tests should now be disabled on sparc64.

Commit https://github.com/codership/galera/commit/0e3e801594802eddeddb7b238286861401c3ace9 also fixes some alignment issues and compiler warnings on Debian testing.

UBSAN reports about undefined behavior in gcomm: https://github.com/codership/galera/commit/bbee99f7fb3a930bc1358e7b7a1602881abfa2fa. This is not absolutely required for build errors reported above, but I included it to get all tests passing when UBSAN is enabled.

SSL test fix: https://github.com/codership/galera/commit/462998ca28e97a95e98e0acb6e6c13cafb95083b, corresponds to https://github.com/codership/galera/issues/647. The needed certificates are now generated automatically before the tests are run, so there should be no issues left with expiration.

Increase verbosity to troubleshoot https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=970043: https://github.com/codership/galera/commit/4433dbf94bbb8318ca5b7b84fbc4f0f070e66377.

For datagram test failures in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1007954: https://github.com/codership/galera/commit/4afcc307ebedf6d90dd799979309b3092d71b76d. The probable reason is that the builder is IPv6 only, and binding to UDP sockets does not work because of that. The test now tries to bind to UDP socket and skips adding test cases for datagram sockets if the bind is not successful.

ottok commented 12 months ago

Thanks @temeo! I filed this as a patch in Debian now in https://salsa.debian.org/mariadb-team/galera-4/-/merge_requests/15 and also running build+testsuite on multiple platforms at https://launchpad.net/~otto/+archive/ubuntu/mariadb/+builds?build_text=&build_state=all to validate there are no regressions.

ottok commented 12 months ago

Btw, you had a missing 'i' in original buffer is still allocated so we need to restore it. If you incorporate codespell --write-changes --summary in your git commit hook it automatically fix minor typos.

ottok commented 12 months ago

Status for https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1053183 / https://github.com/codership/galera/commit/56c6e4a10bd179eb306e9a7cc4abee8f1f309d42 after latest upload:

Sparc64 build+tests failing on:

Test project /<<PKGBUILDDIR>>/obj-sparc64-linux-gnu
    Start 1: gu_tests
1/7 Test #1: gu_tests .........................   Passed    7.25 sec
    Start 2: gu_tests++
2/7 Test #2: gu_tests++ .......................   Passed  127.14 sec
    Start 3: check_gcomm
3/7 Test #3: check_gcomm ......................   Passed   11.33 sec
    Start 4: gcache_tests
4/7 Test #4: gcache_tests .....................   Passed    1.23 sec
    Start 5: gcs_tests
5/7 Test #5: gcs_tests ........................   Passed   25.38 sec
    Start 6: galera_check
6/7 Test #6: galera_check .....................***Failed    7.26 sec
Running suite(s): DataSet
50%: Checks: 2, Failures: 0, Errors: 1
./galera/tests/data_set_check.cpp:220:E:DataSet:ver1:0: (after this point) Received signal 10 (Bus error)
Running suite(s): KeySet
75%: Checks: 4, Failures: 0, Errors: 1
./galera/tests/key_set_check.cpp:93:E:KeySet:ver1_3:0: (after this point) Received signal 10 (Bus error)
Running suite(s): WriteSet
80%: Checks: 5, Failures: 0, Errors: 1
./galera/tests/write_set_ng_check.cpp:101:E:WriteSet basic:ver3_basic_rsv1:0: (after this point) Received signal 10 (Bus error)
Running suite(s): certification
100%: Checks: 6, Failures: 0, Errors: 0
Running suite(s): trx_handle
100%: Checks: 4, Failures: 0, Errors: 0
Running suite(s): service_thd
100%: Checks: 3, Failures: 0, Errors: 0
Running suite(s): ist
100%: Checks: 14, Failures: 0, Errors: 0
Running suite(s): saved_state
100%: Checks: 3, Failures: 0, Errors: 0
Running suite(s): Defaults
100%: Checks: 1, Failures: 0, Errors: 0
Running suite(s): progress_suite
100%: Checks: 1, Failures: 0, Errors: 0
Total tests failed: 3

    Start 7: wsrep_test
7/7 Test #7: wsrep_test .......................   Passed    0.04 sec

ayurchen commented 12 months ago

It appears that I missed to define GALERA_ONLY_ALIGNED for non-x86 architectures, so the tests that access non-aligned data are still executed. Will provide a fix soon.

ayurchen commented 11 months ago

@ottok Tentative (I don't have access to sparc) fix pushed to the repo, please test.

ottok commented 5 months ago

I confirm that sparc64 build was fixed in 26.4.18. From https://buildd.debian.org/status/package.php?p=galera-4&suite=experimental:

ottok commented 2 months ago

FYI Build status for latest Galera-4 26.4.19 upload to Debian (from https://buildd.debian.org/status/package.php?p=galera-4):

codership / galera