Tests segfaults

[l4l]

cmake 3.7.2, clang 3.9.1, libprotoc 3.0.0 (old version from AUR), archlinux build command:

$ cmake .. -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DTESTING=ON && make -j12
$ ctest
Test project /home/kitsu/dev/Work/iroha/build
      Start  1: hash_test
 1/18 Test  #1: hash_test ...........................   Passed    0.99 sec
      Start  2: base64_test
 2/18 Test  #2: base64_test .........................   Passed    0.01 sec
      Start  3: signature_test
 3/18 Test  #3: signature_test ......................   Passed    0.01 sec
      Start  4: java_query_repo_invocation_test
 4/18 Test  #4: java_query_repo_invocation_test .....   Passed    0.07 sec
      Start  5: java_query_repo_account_test
 5/18 Test  #5: java_query_repo_account_test ........   Passed    0.08 sec
      Start  6: java_query_repo_asset_test
 6/18 Test  #6: java_query_repo_asset_test ..........   Passed    0.11 sec
      Start  7: java_query_repo_simple_asset_test
 7/18 Test  #7: java_query_repo_simple_asset_test ...   Passed    0.09 sec
      Start  8: java_query_repo_domain_test
 8/18 Test  #8: java_query_repo_domain_test .........   Passed    0.07 sec
      Start  9: java_query_repo_peer_test
 9/18 Test  #9: java_query_repo_peer_test ...........   Passed    0.09 sec
      Start 10: config_test
10/18 Test #10: config_test .........................   Passed    0.01 sec
      Start 11: http_client_test
11/18 Test #11: http_client_test ....................   Passed    0.01 sec
      Start 12: ensure_sumeragi_json_format_test
12/18 Test #12: ensure_sumeragi_json_format_test ....   Passed    0.08 sec
      Start 13: peer_service_with_json_test
13/18 Test #13: peer_service_with_json_test .........***Exception: SegFault  0.04 sec
      Start 14: java_data_structure_test
14/18 Test #14: java_data_structure_test ............   Passed    0.06 sec
      Start 15: thread_pool_test
15/18 Test #15: thread_pool_test ....................   Passed    1.10 sec
      Start 16: json_test
16/18 Test #16: json_test ...........................   Passed    0.00 sec
      Start 17: connection_grpc_test
17/18 Test #17: connection_grpc_test ................***Exception: SegFault  0.06 sec
      Start 18: transaction_builder_test
18/18 Test #18: transaction_builder_test ............   Passed    0.01 sec

89% tests passed, 2 tests failed out of 18

Total Test time (real) =   2.91 sec

The following tests FAILED:
         13 - peer_service_with_json_test (SEGFAULT)
         17 - connection_grpc_test (SEGFAULT)
Errors while running CTest

valgrind log # 1 valgrind log # 2

[satellitex]

I can correct peer_service_with_json_test segmentation fault. could you wait me.

pull request: peer-service branch

[l4l]

I downgrade the grpc to 1.0.x from 1.1.x and now it's fine (just this cause compile error) Are we considering support of newer grpc branch?

[l4l]

Reopen plz if occur again

[luckychess]

Just got this problem again on my fix/peer-service-config branch (it is exactly the same as develop, only few tests were added for now). It happens not every time so if all tests are OK on your system just try to run them again.

[l4l]

What's about master?

Add a compiling flag -fsanitize=address (or smth similar to your compiler) and if crash happens again report the trace

On 03/10/17 02:58, Konstantin Munichev wrote:

Just got this problem again on my fix/peer-service-config branch (it is exactly the same as develop, only few tests were added for now). It happens not every time so if all tests are OK on your system just try to run them again.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/hyperledger/iroha/issues/190#issuecomment-285524972, or mute the thread https://github.com/notifications/unsubscribe-auth/AFZW43KBPDVSoKQl1I69KL3rJWVtbMSeks5rkJI2gaJpZM4MTDsO.

[luckychess]

@l4l Master is not affected. Log with -fsanitize=address:

/home/luckychess/src/C++/iroha/iroha/cmake-build-debug/test_bin/connection_grpc_test --gtest_filter=* --gtest_color=no
Testing started at 4:03 PM ...
E0310 16:03:19.150921803   32664 server.c:1036]              duplicate registration for /Api.Sumeragi/Torii@*
ASAN:SIGSEGV
=================================================================
==32653==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000038 (pc 0x7f557d2a9a34 bp 0x7f5573266b90 sp 0x7f5573266aa0 T11)
    #0 0x7f557d2a9a33 in pthread_mutex_lock (/lib64/libpthread.so.0+0x9a33)
    #1 0x7f557db751d3 in grpc::Server::Wait() (/usr/local/lib64/libgrpc++.so.1+0x381d3)
    #2 0x5bdba2 in connection::run() /home/luckychess/src/C++/iroha/iroha/core/infra/connection/connection_with_grpc.cpp:599
    #3 0x5b358e in ConnectionWithGrpcTorii_Transaction_Add_Peer_Test::TestBody()::{lambda()#1}::operator()() const (/home/luckychess/src/C++/iroha/iroha/cmake-build-debug/test_bin/connection_grpc_test+0x5b358e)
    #4 0x5b713d in _M_invoke<> /usr/lib/gcc/x86_64-pc-linux-gnu/5.4.0/include/g++-v5/functional:1531
    #5 0x5b6f95 in operator() /usr/lib/gcc/x86_64-pc-linux-gnu/5.4.0/include/g++-v5/functional:1520
    #6 0x5b6ec5 in _M_run /usr/lib/gcc/x86_64-pc-linux-gnu/5.4.0/include/g++-v5/thread:115
    #7 0x7f557cb763e1  (/usr/lib/gcc/x86_64-pc-linux-gnu/5.4.0/libstdc++.so.6+0xdc3e1)
    #8 0x7f557d2a7443  (/lib64/libpthread.so.0+0x7443)
    #9 0x7f557c2d5abc in clone (/lib64/libc.so.6+0xe7abc)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV ??:0 pthread_mutex_lock
Thread T11 created by T0 here:
    #0 0x7f557ddf8603 in pthread_create (/usr/lib/gcc/x86_64-pc-linux-gnu/5.4.0/libasan.so.2+0x36603)
    #1 0x7f557cb76542 in std::thread::_M_start_thread(std::shared_ptr<std::thread::_Impl_base>, void (*)()) (/usr/lib/gcc/x86_64-pc-linux-gnu/5.4.0/libstdc++.so.6+0xdc542)

==32653==ABORTING
Running main() from gtest_main.cc1489150999 DEBUG [config] load json is { 
  "database_path":"/tmp/iroha_ledger",
  "java_class_path":"java_tests",
  "java_class_path_local":"smart_contract/java_tests",
  "java_library_path":"lib",
  "java_library_path_local":"build/lib",
  "java_policy_path":"jvm/java.policy.txt",
  "concurrency": 0,
  "max_faulty_peers" : 1,
  "pool_worker_queue_size": 1024,
  "http_port": 1204,
  "grpc_port": 50051
}

1489150999 DEBUG [config] load json is {
  "group": [
    {
      "ip": "127.0.0.1",
      "name": "da77880a3da4",
      "publicKey": "u7X/zQ/Dq21WW7YH4rbkpiCYJXjPxk5t3qNDKiVwBx8="
    }
  ],
  "me": {
    "ip": "127.0.0.1",
    "name": "da77880a3da4",
    "privateKey": "cPY84e0BXGUHBjT4QdlPI0LI3BPIfUfSZjB8jdWURkNQ+pEagT/ysrewbt2YUo/Qbfd5vczW5oDooGSNUBTj9g==",
    "publicKey": "u7X/zQ/Dq21WW7YH4rbkpiCYJXjPxk5t3qNDKiVwBx8="
  }
}
1489150999 INFO [connection] Operation
1489150999 INFO [connection] size: 0
1489150999 INFO [connection] name: 
1489150999 INFO [connection] size: 0
type: "Add"
senderPubkey: "karin"
domain {
  ownerPublicKey: "pubkey1"
  name: "name"
}

1489150999 INFO [connection] response: OK
Process finished with exit code 1

[luckychess]

I think I got it finally: connection_grpc_test starts thread for grpc server for both test cases but after the first test case finished thread still remains alive. This usually leads to problems when the second test case starts. Unfortunately std::thread has no way to terminate it so I found a workaround: move second test case to another file to make it run in another process. https://github.com/hyperledger/iroha/pull/210 @l4l Please check (or somebody else). May be better solutions exist.

[l4l]

fixed in #210