search

openucx / ucx

Unified Communication X (mailing list - https://elist.ornl.gov/mailman/listinfo/ucx-group)
http://www.openucx.org
Other
1.17k stars 428 forks source link

TCP sockcm received wrong data length #6740

Open yosefe opened 3 years ago

yosefe commented 3 years ago

https://dev.azure.com/ucfconsort/0b36e3f0-8ab9-4a48-b68b-4b2350e02c88/_apis/build/builds/17255/logs/319

2021-04-30T20:53:39.5477011Z [ RUN      ] tcp/test_uct_sockaddr.err_handle/17 </bond0>
2021-04-30T20:53:39.5477811Z [     INFO ] Testing tcp on :::36126 interface bond0
2021-04-30T20:53:39.5479360Z /scrap/azure/agent-03/AZP_WORKSPACE/1/s/contrib/../test/gtest/uct/ib/test_sockaddr.cc:316: Failure
2021-04-30T20:53:39.5480281Z wrong data length received 0
2021-04-30T20:53:39.5481511Z terminate called after throwing an instance of 'ucs::test_abort_exception'
2021-04-30T20:53:39.5482395Z   what():  std::exception
2021-04-30T20:53:39.5483207Z [sputnik4:9467 :async:1294] Caught signal 6 (Aborted: tkill(2) or tgkill(2))
2021-04-30T20:53:39.5484048Z ==== backtrace (tid:   1294) ====
2021-04-30T20:53:39.5484740Z  0 0x0000000000036207 __GI_raise()  :0
2021-04-30T20:53:39.5485692Z  1 0x00000000000378f8 __GI_abort()  :0
2021-04-30T20:53:39.5486513Z  2 0x00000000000607d5 __gnu_cxx::__verbose_terminate_handler()  ???:0
2021-04-30T20:53:39.5487371Z  3 0x000000000005e746 std::rethrow_exception()  ???:0
2021-04-30T20:53:39.5488136Z  4 0x000000000005e773 std::terminate()  ???:0
2021-04-30T20:53:39.5488874Z  5 0x000000000005e993 __cxa_throw()  ???:0
2021-04-30T20:53:39.5490384Z  6 0x0000000000ac8f41 test_uct_sockaddr::verify_remote_data()  /scrap/azure/agent-03/AZP_WORKSPACE/1/s/contrib/../test/gtest/uct/ib/test_sockaddr.cc:316
2021-04-30T20:53:39.5492384Z  7 0x0000000000ac9a7a test_uct_sockaddr::client_connect_cb()  /scrap/azure/agent-03/AZP_WORKSPACE/1/s/contrib/../test/gtest/uct/ib/test_sockaddr.cc:434
2021-04-30T20:53:39.5494267Z  8 0x0000000000ac9a7a test_uct_sockaddr::client_connect_cb()  /scrap/azure/agent-03/AZP_WORKSPACE/1/s/contrib/../test/gtest/uct/ib/test_sockaddr.cc:436
2021-04-30T20:53:39.5496041Z  9 0x0000000000016a75 uct_cm_ep_client_connect_cb()  /scrap/azure/agent-03/AZP_WORKSPACE/1/s/contrib/../src/uct/base/uct_cm.c:122
2021-04-30T20:53:39.5497769Z 10 0x000000000002f776 uct_tcp_sockcm_ep_client_connect_cb()  /scrap/azure/agent-03/AZP_WORKSPACE/1/s/contrib/../src/uct/tcp/tcp_sockcm_ep.c:52
2021-04-30T20:53:39.5499562Z 11 0x000000000002f776 uct_tcp_sockcm_ep_client_invoke_connect_cb()  /scrap/azure/agent-03/AZP_WORKSPACE/1/s/contrib/../src/uct/tcp/tcp_sockcm_ep.c:636
2021-04-30T20:53:39.5501410Z 12 0x000000000002f776 uct_tcp_sockcm_ep_handle_data_received()  /scrap/azure/agent-03/AZP_WORKSPACE/1/s/contrib/../src/uct/tcp/tcp_sockcm_ep.c:693
2021-04-30T20:53:39.5503140Z 13 0x000000000002fac0 uct_tcp_sockcm_ep_recv()  /scrap/azure/agent-03/AZP_WORKSPACE/1/s/contrib/../src/uct/tcp/tcp_sockcm_ep.c:774
2021-04-30T20:53:39.5504778Z 14 0x000000000002d1e8 uct_tcp_sa_data_handler()  /scrap/azure/agent-03/AZP_WORKSPACE/1/s/contrib/../src/uct/tcp/tcp_sockcm.c:112
2021-04-30T20:53:39.5506431Z 15 0x0000000000047a4a ucs_async_handler_invoke()  /scrap/azure/agent-03/AZP_WORKSPACE/1/s/contrib/../src/ucs/async/async.c:249
2021-04-30T20:53:39.5508078Z 16 0x0000000000047a4a ucs_async_dispatch_handlers()  /scrap/azure/agent-03/AZP_WORKSPACE/1/s/contrib/../src/ucs/async/async.c:303
2021-04-30T20:53:39.5509738Z 17 0x000000000004b37c ucs_async_thread_ev_handler()  /scrap/azure/agent-03/AZP_WORKSPACE/1/s/contrib/../src/ucs/async/thread.c:87
2021-04-30T20:53:39.5511446Z 18 0x000000000006d651 ucs_event_set_wait()  /scrap/azure/agent-03/AZP_WORKSPACE/1/s/contrib/../src/ucs/sys/event_set.c:215
2021-04-30T20:53:39.5513074Z 19 0x000000000004bc03 ucs_async_thread_func()  /scrap/azure/agent-03/AZP_WORKSPACE/1/s/contrib/../src/ucs/async/thread.c:130
2021-04-30T20:53:39.5514127Z 20 0x0000000000007dd5 start_thread()  pthread_create.c:0
2021-04-30T20:53:39.5514909Z 21 0x00000000000fdead __clone()  ???:0
2021-04-30T20:53:39.5515783Z =================================
2021-04-30T20:53:39.5516495Z [sputnik4:9467 :async:1294] Process frozen...
2021-04-30T23:17:06.4936003Z make: *** [test] Terminated
2021-04-30T23:17:06.5073857Z ##[error]Bash exited with code '124'.
2021-04-30T23:17:06.5098427Z ##[section]Finishing: Run ./contrib/test_jenkins.sh
evgeny-leksikov commented 3 years ago

seems like duplicate of https://github.com/openucx/ucx/issues/6451