Blockstream / gdk

Blockstream Green Development Kit
Other
159 stars 58 forks source link

GDK 0.68.0 Windows build crashes in `GA_connect` #198

Closed pavel-kokolemin closed 11 months ago

pavel-kokolemin commented 1 year ago

GDK 0.68.0 Windows build crashes in GA_connect when used with multi-sig accounts (single-sig accounts work fine).

I have reproduced the crash in a minimal program (main.cpp). To build:

x86_64-w64-mingw32-c++ main.cpp -o gdk_crash -I/root/gdk/include -I/root/gdk/build-windows-mingw-w64/src -I/prebuild/mingw-w64/nlohmann_json/include /root/gdk/build-windows-mingw-w64/libgreenaddress.dll.a

Copy libgreenaddress.dll and required mingw libs:

/root/gdk/build-windows-mingw-w64/libgreenaddress.dll
/usr/lib/gcc/x86_64-w64-mingw32/10-posix/libssp-0.dll
/usr/lib/gcc/x86_64-w64-mingw32/10-posix/libstdc++-6.dll
/usr/lib/gcc/x86_64-w64-mingw32/10-posix/libgcc_s_seh-1.dll
/usr/x86_64-w64-mingw32/lib/libwinpthread-1.dll

Then run the binary (gdk_crash.exe):

GA_init...
send: {"datadir":"./","log_level":"debug"}
34.413 DEBUG - populated the cache with 0 files
34.413 DEBUG - loading 0 cache files
34.414 INFO - `init` took 4.43793ms
GA_create_session...
GA_connect...
send: {"name":"liquid"}
[2023-10-05 09:27:34.409299] [0x00001688] [info]    change_state_to: requesting state connected
[2023-10-05 09:27:34.409299] [0x00002204] [info]    net: thread started for gdk version 
[2023-10-05 09:27:34.426265] [0x00002204] [info]    net: desired connected actual disconnected
[2023-10-05 09:27:34.426265] [0x00002204] [info]    net: connect to wss://green-liquid-mainnet.blockstream.com/v2/ws
[2023-10-05 09:27:39.444271] [0x000028c4] [info]    Error getting remote endpoint: system:10009 (The file handle supplied is not valid)
[2023-10-05 09:27:39.459714] [0x000028c4] [info]    asio async_shutdown error: asio.ssl:336462100 (asio.ssl error)
[2023-10-05 09:27:39.459714] [0x00002204] [info]    transport connect ignoring exception:Dynamic exception type: boost::wrapexcept<autobahn::network_error>
std::exception::what: failed to connect
[2023-10-05 09:27:39.475348] [0x00002204] [info]    ignoring exception:Dynamic exception type: boost::wrapexcept<std::exception>
std::exception::what: std::exception
[2023-10-05 09:27:39.475348] [0x00002204] [info]    ignoring exception:Dynamic exception type: boost::wrapexcept<autobahn::protocol_error>
std::exception::what: session already stopped
[2023-10-05 09:27:39.490892] [0x00002204] [info]    ignoring exception:network transport already disconnected
[2023-10-05 09:27:39.490892] [0x00002204] [debug]   waiting for connection to die
[2023-10-05 09:27:39.506534] [0x00002204] [debug]   connection is dead
[2023-10-05 09:27:39.506534] [0x00002204] [info]    net: backing off for 2s
Segmentation fault

I tried building on Debian 12 and Debian 11, same crash happens. It works fine on Linux:

GA_init...
send: {"datadir":"./","log_level":"debug"}
11.086 DEBUG - populated the cache with 0 files
11.086 DEBUG - loading 0 cache files
11.086 INFO - `init` took 279.464µs
GA_create_session...
GA_connect...
send: {"name":"liquid"}
[2023-10-05 10:40:11.088242] [0x00007fb3000bc640] [debug]   endpoint constructor
[2023-10-05 10:40:11.088272] [0x00007fb3000bc640] [debug]   client constructor
[2023-10-05 10:40:11.088307] [0x00007fb3000bc640] [debug]   set_pong_timeout_handler
[2023-10-05 10:40:11.088319] [0x00007fb2f87216c0] [info]    net: thread started for gdk version 
[2023-10-05 10:40:11.088335] [0x00007fb3000bc640] [debug]   asio::init_asio
[2023-10-05 10:40:11.088582] [0x00007fb3000bc640] [info]    change_state_to: requesting state connected
[2023-10-05 10:40:11.088610] [0x00007fb2f87216c0] [info]    net: desired connected actual disconnected
[2023-10-05 10:40:11.088617] [0x00007fb2f87216c0] [info]    net: connect to wss://green-liquid-mainnet.blockstream.com/v2/ws
[2023-10-05 10:40:11.088711] [0x00007fb2f87216c0] [debug]   set_open_handler
[2023-10-05 10:40:11.088721] [0x00007fb2f87216c0] [debug]   set_close_handler
[2023-10-05 10:40:11.088725] [0x00007fb2f87216c0] [debug]   set_fail_handler
[2023-10-05 10:40:11.088735] [0x00007fb2f87216c0] [debug]   set_message_handler
[2023-10-05 10:40:11.088846] [0x00007fb2f87216c0] [debug]   create_connection
[2023-10-05 10:40:11.088922] [0x00007fb2f87216c0] [debug]   asio con transport constructor
[2023-10-05 10:40:11.089015] [0x00007fb2f87216c0] [debug]   connection constructor
[2023-10-05 10:40:11.089078] [0x00007fb2f87216c0] [debug]   transport::asio::init
[2023-10-05 10:40:11.090339] [0x00007fb2f87216c0] [debug]   starting async DNS resolve for green-liquid-mainnet.blockstream.com:443
[2023-10-05 10:40:11.262926] [0x00007fb2f8f226c0] [debug]   Async DNS resolve successful. Results: 34.110.237.101:443 
[2023-10-05 10:40:11.263000] [0x00007fb2f8f226c0] [debug]   Starting async connect
[2023-10-05 10:40:11.263777] [0x00007fb2f8f226c0] [debug]   asio handle_resolve_timeout timer cancelled
[2023-10-05 10:40:11.393245] [0x00007fb2f8f226c0] [debug]   Async connect to 34.110.237.101:443 successful.
[2023-10-05 10:40:11.393367] [0x00007fb2f8f226c0] [debug]   connection start
[2023-10-05 10:40:11.393451] [0x00007fb2f8f226c0] [debug]   asio connection init
[2023-10-05 10:40:11.393601] [0x00007fb2f8f226c0] [debug]   asio connection handle pre_init
[2023-10-05 10:40:11.393617] [0x00007fb2f8f226c0] [debug]   asio connection post_init
[2023-10-05 10:40:11.395008] [0x00007fb2f8f226c0] [debug]   asio handle_connect_timeout timer cancelled
[2023-10-05 10:40:11.526730] [0x00007fb2f8f226c0] [debug]   Checking for pinned certificate
[2023-10-05 10:40:11.526924] [0x00007fb2f8f226c0] [debug]   Found pinned certificate 64e286b76063602a372efd60cde8db2656a49ee15e84254b3d6eb5fe38f4288b
[2023-10-05 10:40:11.527622] [0x00007fb2f8f226c0] [debug]   asio connection handle_post_init
[2023-10-05 10:40:11.527676] [0x00007fb2f8f226c0] [debug]   connection handle_transport_init
[2023-10-05 10:40:11.527824] [0x00007fb2f8f226c0] [debug]   connection send_http_request
[2023-10-05 10:40:11.528297] [0x00007fb2f8f226c0] [debug]   Raw Handshake request:
GET /v2/ws HTTP/1.1
Connection: Upgrade
Host: green-liquid-mainnet.blockstream.com
Sec-WebSocket-Key: KLziMDuB262iqt5fuv6PPQ==
Sec-WebSocket-Protocol: wamp.2.msgpack
Sec-WebSocket-Version: 13
Upgrade: websocket
User-Agent: WebSocket++/0.8.1

[2023-10-05 10:40:11.529058] [0x00007fb2f8f226c0] [debug]   asio post init timer cancelled
[2023-10-05 10:40:11.529366] [0x00007fb2f8f226c0] [debug]   handle_send_http_request
[2023-10-05 10:40:11.529410] [0x00007fb2f8f226c0] [debug]   asio async_read_at_least: 1
[2023-10-05 10:40:11.829420] [0x00007fb2f8f226c0] [debug]   asio con handle_async_read
[2023-10-05 10:40:11.829552] [0x00007fb2f8f226c0] [debug]   handle_read_http_response
[2023-10-05 10:40:11.829968] [0x00007fb2f8f226c0] [debug]   Raw response: HTTP/1.1 101 Switching Protocols
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
Connection: Upgrade
Date: Thu, 05 Oct 2023 06:40:11 GMT
Sec-WebSocket-Accept: obqBne+fO1l0JKwY1nVxYdO6oQo=
Sec-WebSocket-Protocol: wamp.2.msgpack
Server: openresty/1.21.4.1
Upgrade: websocket
Via: 1.1 google

[2023-10-05 10:40:11.830353] [0x00007fb2f8f226c0] [debug]   p = 0 bytes transferred = 0
[2023-10-05 10:40:11.830379] [0x00007fb2f8f226c0] [debug]   asio async_read_at_least: 1
[2023-10-05 10:40:11.830524] [0x00007fb2f8f226c0] [debug]   open handshake timer cancelled
[2023-10-05 10:40:11.831771] [0x00007fb2f8f226c0] [debug]   connection send
[2023-10-05 10:40:11.831972] [0x00007fb2f8f226c0] [debug]   write_push: message count: 1 buffer size: 125
[2023-10-05 10:40:11.832703] [0x00007fb2f8f226c0] [debug]   write_pop: message count: 0 buffer size: 0
[2023-10-05 10:40:11.833118] [0x00007fb2f8f226c0] [debug]   connection handle_write_frame
[2023-10-05 10:40:12.000661] [0x00007fb2f8f226c0] [debug]   asio con handle_async_read
[2023-10-05 10:40:12.000807] [0x00007fb2f8f226c0] [debug]   p = 0 bytes transferred = 918
[2023-10-05 10:40:12.000827] [0x00007fb2f8f226c0] [debug]   calling consume with 918 bytes
[2023-10-05 10:40:12.001189] [0x00007fb2f8f226c0] [debug]   Processing Bytes: 82 7E 03 92 93 02 CF 00 01 0D 68 83 2D 39 C3 8B A9 78 5F 63 62 5F 6E 6F 64 65 D9 2D 6C 69 71 75 69 64 2D 6D 61 69 6E 6E 65 74 2D 62 61 63 6B 65 6E 64 2D 65 75 72 6F 70 65 2D 77 65 73 74 32 2D 76 37 37 63 2D 32 34 38 32 AB 78 5F 63 62 5F 77 6F 72 6B 65 72 A9 77 6F 72 6B 65 72 30 30 31 A9 78 5F 63 62 5F 70 65 65 72 B4 74 63 70 34 3A 31 32 37 2E 30 2E 30 2E 31 3A 33 32 38 31 30 A8 78 5F 63 62 5F 70 69 64 CD 09 E1 A5 72 65 61 6C 6D A6 72 65 61 6C 6D 31 A6 61 75 74 68 69 64 BD 34 4D 53 50 2D 33 50 54 57 2D 59 54 46 4D 2D 47 48 46 48 2D 4C 57 35 37 2D 45 4E 43 59 A8 61 75 74 68 72 6F 6C 65 A9 61 6E 6F 6E 79 6D 6F 75 73 AA 61 75 74 68 6D 65 74 68 6F 64 A9 61 6E 6F 6E 79 6D 6F 75 73 AC 61 75 74 68 70 72 6F 76 69 64 65 72 A7 64 79 6E 61 6D 69 63 A9 61 75 74 68 65 78 74 72 61 84 A9 78 5F 63 62 5F 6E 6F 64 65 D9 2D 6C 69 71 75 69 64 2D 6D 61 69 6E 6E 65 74 2D 62 61 63 6B 65 6E 64 2D 65 75 72 6F 70 65 2D 77 65 73 74 32 2D 76 37 37 63 2D 32 34 38 32 AB 78 5F 63 62 5F 77 6F 72 6B 65 72 A9 77 6F 72 6B 65 72 30 30 31 A9 78 5F 63 62 5F 70 65 65 72 B4 74 63 70 34 3A 31 32 37 2E 30 2E 30 2E 31 3A 33 32 38 31 30 A8 78 5F 63 62 5F 70 69 64 CD 09 E1 A5 72 6F 6C 65 73 82 A6 62 72 6F 6B 65 72 81 A8 66 65 61 74 75 72 65 73 8A B8 70 75 62 6C 69 73 68 65 72 5F 69 64 65 6E 74 69 66 69 63 61 74 69 6F 6E C3 BA 70 61 74 74 65 72 6E 5F 62 61 73 65 64 5F 73 75 62 73 63 72 69 70 74 69 6F 6E C3 B0 73 65 73 73 69 6F 6E 5F 6D 65 74 61 5F 61 70 69 C3 B5 73 75 62 73 63 72 69 70 74 69 6F 6E 5F 6D 65 74 61 5F 61 70 69 C3 BD 73 75 62 73 63 72 69 62 65 72 5F 62 6C 61 63 6B 77 68 69 74 65 5F 6C 69 73 74 69 6E 67 C3 B3 70 75 62 6C 69 73 68 65 72 5F 65 78 63 6C 75 73 69 6F 6E C3 B7 73 75 62 73 63 72 69 70 74 69 6F 6E 5F 72 65 76 6F 63 61 74 69 6F 6E C3 B4 70 61 79 6C 6F 61 64 5F 74 72 61 6E 73 70 61 72 65 6E 63 79 C3 BC 70 61 79 6C 6F 61 64 5F 65 6E 63 72 79 70 74 69 6F 6E 5F 63 72 79 70 74 6F 62 6F 78 C3 AF 65 76 65 6E 74 5F 72 65 74 65 6E 74 69 6F 6E C3 A6 64 65 61 6C 65 72 81 A8 66 65 61 74 75 72 65 73 8B B5 63 61 6C 6C 65 72 5F 69 64 65 6E 74 69 66 69 63 61 74 69 6F 6E C3 BA 70 61 74 74 65 72 6E 5F 62 61 73 65 64 5F 72 65 67 69 73 74 72 61 74 69 6F 6E C3 B0 73 65 73 73 69 6F 6E 5F 6D 65 74 61 5F 61 70 69 C3 B5 72 65 67 69 73 74 72 61 74 69 6F 6E 5F 6D 65 74 61 5F 61 70 69 C3 B3 73 68 61 72 65 64 5F 72 65 67 69 73 74 72 61 74 69 6F 6E C3 AE 63 61 6C 6C 5F 63 61 6E 63 65 6C 69 6E 67 C3 B8 70 72 6F 67 72 65 73 73 69 76 65 5F 63 61 6C 6C 5F 72 65 73 75 6C 74 73 C3 B7 72 65 67 69 73 74 72 61 74 69 6F 6E 5F 72 65 76 6F 63 61 74 69 6F 6E C3 B4 70 61 79 6C 6F 61 64 5F 74 72 61 6E 73 70 61 72 65 6E 63 79 C3 B2 74 65 73 74 61 6D 65 6E 74 5F 6D 65 74 61 5F 61 70 69 C3 BC 70 61 79 6C 6F 61 64 5F 65 6E 63 72 79 70 74 69 6F 6E 5F 63 72 79 70 74 6F 62 6F 78 C3 
[2023-10-05 10:40:12.001295] [0x00007fb2f8f226c0] [debug]   bytes left after consume: 0
[2023-10-05 10:40:12.001308] [0x00007fb2f8f226c0] [debug]   Complete message received. Dispatching
[2023-10-05 10:40:12.004569] [0x00007fb2f8f226c0] [debug]   asio async_read_at_least: 1
[2023-10-05 10:40:12.004737] [0x00007fb2f87216c0] [info]    net: connection successful
[2023-10-05 10:40:12.041028] [0x00007fb3000bc640] [info]    change_state_to: changed to connected
Success

main.cpp.txt

jgriffiths commented 1 year ago

Hi @pavel-kokolemin thanks for the report.

Can you confirm if this works for you with an earlier gdk version, and if so what is the last version that works?

pavel-kokolemin commented 1 year ago

Can you confirm if this works for you with an earlier gdk version, and if so what is the last version that works?

I have tried release_0.67.1 and some older versions and they do not crash:

GA_init...
send: {"datadir":"./","log_level":"debug"}
06.852 DEBUG - populated the cache with 0 files
06.852 DEBUG - loading 0 cache files
06.852 INFO - `init` took 973.43µs
GA_create_session...
GA_connect...
send: {"name":"liquid"}
[2023-10-05 12:02:06.851755] [0x00000cf0] [info]    net: thread started for gdk version
[2023-10-05 12:02:06.851755] [0x00000ba8] [info]    change_state_to: requesting state connected
[2023-10-05 12:02:06.867991] [0x00000cf0] [info]    net: desired connected actual disconnected
[2023-10-05 12:02:06.867991] [0x00000cf0] [info]    net: connect to wss://green-liquid-mainnet.blockstream.com/v2/ws
[2023-10-05 12:02:06.978625] [0x00000570] [debug]   Checking for pinned certificate
[2023-10-05 12:02:06.994263] [0x00000570] [debug]   Found pinned certificate 64e286b76063602a372efd60cde8db2656a49ee15e84254b3d6eb5fe38f4288b
TX message (125 octets) ...
TX message: hello ["realm1", {"authid":"","authmethods":[],"roles":{"subscriber":{},"publisher":{},"callee":{"features":{"call_timeout":true}},"caller":{"features":{"call_timeout":true}}}}]      
RX message received.
RX message: welcome [8796734805482752, {"x_cb_node":"liquid-mainnet-backend-europe-west2-v77c-2482","x_cb_worker":"worker001","x_cb_peer":"tcp4:127.0.0.1:40206","x_cb_pid":2529,"realm":"realm1","authid":"WXFV-EPA6-VH4U-XRA9-GC5V-FG3K","authrole":"anonymous","authmethod":"anonymous","authprovider":"dynamic","authextra":{"x_cb_node":"liquid-mainnet-backend-europe-west2-v77c-2482","x_cb_worker":"worker001","x_cb_peer":"tcp4:127.0.0.1:40206","x_cb_pid":2529},"roles":{"broker":{"features":{"publisher_identification":true,"pattern_based_subscription":true,"session_meta_api":true,"subscription_meta_api":true,"subscriber_blackwhite_listing":true,"publisher_exclusion":true,"subscription_revocation":true,"payload_transparency":true,"payload_encryption_cryptobox":true,"event_retention":true}},"dealer":{"features":{"caller_identification":true,"pattern_based_registration":true,"session_meta_api":true,"registration_meta_api":true,"shared_registration":true,"call_canceling":true,"progressive_call_results":true,"registration_revocation":true,"payload_transparency":true,"testament_meta_api":true,"payload_encryption_cryptobox":true}}}}]
[2023-10-05 12:02:07.295945] [0x00000cf0] [info]    net: connection successful
[2023-10-05 12:02:07.376252] [0x00000ba8] [info]    change_state_to: changed to connected
Success

So it looks like a regression in release_0.68.0.

borgbyte commented 1 year ago

@pavel-kokolemin can you try branch release_0.68_dev?

pavel-kokolemin commented 1 year ago

@pavel-kokolemin can you try branch release_0.68_dev?

@borgbyte work fine, thanks.

borgbyte commented 1 year ago

@pavel-kokolemin fyi 0.68.1 is in pre-release

lightyear15 commented 12 months ago

Hello @pavel-kokolemin

even though we managed to fix your windows issue by reverting some changes, we are still investigating the root cause of the crash.

We are having hard times replicating it though. May I ask some questions?

As last request, can you please retest version 0.68.0 with your main.cpp file changed like this?

--- main.cpp.txt    2023-10-25 10:10:51.859571074 +0200
+++ main.cpp2.txt   2023-10-25 10:15:11.635713340 +0200
@@ -31,7 +31,8 @@

 int main() {
     std::cout << "GA_init..." << std::endl;
-    check(GA_init(json({{"datadir", "./"}, {"log_level", "debug"}})));
+    auto *jason = json({{"datadir", "./"}, {"log_level", "debug"}});
+    check(GA_init(jason));

     std::cout << "GA_create_session..." << std::endl;
     GA_session *session;
@@ -43,5 +44,8 @@
     std::cout << "GA_connect..." << std::endl;
     check(GA_connect(session, json({{"name", "liquid"}})));

+    check(GA_destroy_json(jason));
+    check(GA_destroy_session(session));
+
     std::cout << "Success" << std::endl;
 }

thanks

pavel-kokolemin commented 12 months ago

Hi @lightyear15

what windows version were you using?

I'm using Windows 11. The first time the crash was reported by our test user, so I doubt it's related to my Windows installation. And why do other builds work?

do you remember your machine having some connectivity problems?

No. The crash always happens and I have a good network connection.

can you please retest version 0.68.0 with your main.cpp file changed like this?

I applied the patch and tried 0.68.0 and the latest master branch (commit 5021276cbc3c74c52946aa7450a76ac9db5a63f4) and it still crashes. 0.68.2 works fine as I already reported.

We are having hard times replicating it though.

Can you send your libgreenaddress.dll build?

Thanks.

lightyear15 commented 12 months ago

It seems on Windows10 the bug does not pop up.

The zip file attached contains the libgreenaddress.dll and the executable result of the build of my version of your main.cpp (the one patched according to comment above + some few more logs) We tested this binary with windows11 as well and no crash has been reported gdkwincrash.zip

pavel-kokolemin commented 12 months ago

I can't reproduce the crash either. But it crashes:

While the GDK build without 52f2b2eb074fa7f64619d08332d83f47c75a51bf commit works normally in all situations. I will do some more testing and get back to you. Thanks for your help.

lightyear15 commented 11 months ago

I can't reproduce the crash either. But it crashes:

* When running gdkcrash.exe from the bash shell (I use Git for Windows).

* In our wallet application.

While the GDK build without 52f2b2e commit works normally in all situations. I will do some more testing and get back to you. Thanks for your help.

Thanks @pavel-kokolemin for your help:

I am not sure I understand the situation:

can you give me some insight on what your build instructions are?

Thanks

pavel-kokolemin commented 11 months ago

Hi @lightyear15 Sorry for the delay. Please try to reproduce the crash with this: main.cpp.txt It crashes reliably for me.

I built it as before: GDK:

./tools/build.sh --mingw-w64

And gdk_crash.exe:

x86_64-w64-mingw32-c++ main.cpp -o gdk_crash -I/path/to/include/gdk -I/path/to/gdk/build-windows-mingw-w64/external_deps/nlohmann_json/include /path/to/gdk/build-windows-mingw-w64/libgreenaddress.dll.a

Copied gdk_crash.exe, libgreenaddress.dll and these mingw libs (from Debian 12):

/usr/lib/gcc/x86_64-w64-mingw32/12-posix/libstdc++-6.dll
/usr/lib/gcc/x86_64-w64-mingw32/12-posix/libgcc_s_seh-1.dll
/usr/x86_64-w64-mingw32/lib/libwinpthread-1.dll

I have tried: latest master (with commit 52f2b2eb074fa7f64619d08332d83f47c75a51bf): crash release_0.68.0 (with commit 52f2b2eb074fa7f64619d08332d83f47c75a51bf): crash release_0.68.4 (without commit 52f2b2eb074fa7f64619d08332d83f47c75a51bf): no crash

The same crash happened when I tried mingw from Debian 11.

lightyear15 commented 11 months ago

Hi @pavel-kokolemin

can you please do one more round of tests using gdk from this branch? https://github.com/Blockstream/gdk/tree/giulio_mingw_fix, commit 357ceb3b38df59f9904b60301611a3a6b21bf6e8

Thanks a lot for your invaluable help in testing and investigating this issue. I hope this will finally put an end to this issue :crossed_fingers:

pavel-kokolemin commented 11 months ago

Hi @lightyear15, thank you!

I tried this commit and it works a bit better (no crash after the first iteration), but it still crashes after about 10..20 iterations. And I checked release_0.68.4 again and there are no crashes after all 100 iterations.

lightyear15 commented 11 months ago

Hi @pavel-kokolemin , I promise, this will be the final one. same branch, updated to b620464ae780e387603d9866e9731922abb801a0

:crossed_fingers: :crossed_fingers:

pavel-kokolemin commented 11 months ago

@lightyear15 No problem! Sorry, I don't see the commit b620464ae780e387603d9866e9731922abb801a0. I tried b7d4e1454ba04af702eb9b93d6d717c5cd4799b4 but it still crashes.

lightyear15 commented 11 months ago

@pavel-kokolemin yes sorry, you're pointing to the correct commit, my bad.

During the whole process of debugging this issue I managed to reproduce the bug as you were reporting. I could also reproduce the issue you reported where the crash was happening after 10, 20 iterations, so I am pretty confident that we were experiencing the same bug. Now that in my set-up your 100-iteration test does not crash, not even a single time (neither release nor debug builds), I am inclined to think that there is something not right in your set-up. For instance, we upgraded boost to 1.82.0, so be sure to recompile all the dependencies.

Thanks a lot again

pavel-kokolemin commented 11 months ago

I checked again, b7d4e1454ba04af702eb9b93d6d717c5cd4799b4 crashes after about 10..20 iterations while release_0.68.4 works normally. And I have rebuilt all dependencies in both cases.

lightyear15 commented 11 months ago

hello @pavel-kokolemin , I definitely lost you here :disappointed:
I cannot reproduce the bug you're reporting any more.

May I ask you to report a stack trace of the bug? A few hints on how to get it:

let me know if you need more detailed explanation or any further help on how to get a stack trace of the bug.

Cheers

pavel-kokolemin commented 11 months ago

Now I can't reproduce the crash either. I guess something was wrong on my end. I will close the issue. Thanks for your help!