search

comit-network / xmr-btc-swap

Bitcoin–Monero Cross-chain Atomic Swap
GNU General Public License v3.0
645 stars 79 forks source link

Monero container startup fails e2e tests #526

Open da-kami opened 3 years ago

da-kami commented 3 years ago

Here is a run where the build fails due to the monero wallet RPC not being available properly: https://github.com/comit-network/xmr-btc-swap/runs/2652333552

It is unclear what exactly causes this, but from the logs it looks like the monero-wallet-rpc is not fully started yet when we send requests to it:

 May 24 01:56:21.364  INFO monero_harness: Starting monerod: wlku_monerod
May 24 01:56:29.079  INFO testcontainers::core::wait_for_message: Found message after comparing 183 lines    
May 24 01:56:29.135  INFO monero_harness: Starting miner wallet: miner
May 24 01:56:31.454  INFO testcontainers::core::wait_for_message: Found message after comparing 16 lines    
May 24 01:56:34.033  INFO monero_harness: Starting wallet: alice
May 24 01:56:36.558  INFO testcontainers::core::wait_for_message: Found message after comparing 16 lines    
thread 'alice_manually_punishes_after_bob_dead' panicked at 'called `Result::unwrap()` on an `Err` value: error sending request for url (http://127.0.0.1:49177/json_rpc): operation was canceled: connection closed before message completed

Caused by:
    0: error sending request for url (http://127.0.0.1:49177/json_rpc): operation was canceled: connection closed before message completed
    1: operation was canceled: connection closed before message completed
    2: connection closed before message completed', swap/tests/harness/mod.rs:157:14

It might be worth experimenting with different messages to wait for on the monero-wallet-rpc side. Recently I have not recorded failures where waiting for monerod caused issues, but only the wallet RPC.

da-kami commented 3 years ago

I did not encounter this problem recently, so I assume that https://github.com/comit-network/xmr-btc-swap/pull/542 "fixed" it. @thomaseizinger @rishflab if you have encountered this please re-open.

da-kami commented 3 years ago

Unfortunately have to reopen, just ran into this here: https://github.com/comit-network/xmr-btc-swap/pull/581/checks?check_run_id=2882307126

 Jun 22 07:13:46.712  INFO testcontainers::core::wait_for_message: Found message after comparing 77 lines    
Jun 22 07:13:51.678  INFO testcontainers::core::wait_for_message: Found message after comparing 2 lines    
Jun 22 07:13:51.678  INFO monero_harness: Starting monerod: 7YBr_monerod
Jun 22 07:13:58.668  INFO testcontainers::core::wait_for_message: Found message after comparing 183 lines    
Jun 22 07:13:58.719  INFO monero_harness: Starting miner wallet: miner
Jun 22 07:14:02.138  INFO testcontainers::core::wait_for_message: Found message after comparing 16 lines    
Jun 22 07:14:03.519  INFO monero_harness: Starting wallet: alice
Jun 22 07:14:05.683  INFO testcontainers::core::wait_for_message: Found message after comparing 16 lines    
Jun 22 07:14:05.737  WARN monero_harness: Monero wallet RPC emitted error error sending request for url (http://127.0.0.1:49177/json_rpc): operation was canceled: connection closed before message completed - retrying to create wallet in 2 seconds...
Jun 22 07:14:07.740  WARN monero_harness: Monero wallet RPC emitted error error sending request for url (http://127.0.0.1:49177/json_rpc): operation was canceled: connection closed before message completed - retrying to create wallet in 2 seconds...
Jun 22 07:14:09.743  WARN monero_harness: Monero wallet RPC emitted error error sending request for url (http://127.0.0.1:49177/json_rpc): operation was canceled: connection closed before message completed - retrying to create wallet in 2 seconds...
Jun 22 07:14:11.747  WARN monero_harness: Monero wallet RPC emitted error error sending request for url (http://127.0.0.1:49177/json_rpc): operation was canceled: connection closed before message completed - retrying to create wallet in 2 seconds...
Jun 22 07:14:13.750  WARN monero_harness: Monero wallet RPC emitted error error sending request for url (http://127.0.0.1:49177/json_rpc): connection closed before message completed - retrying to create wallet in 2 seconds...
thread 'given_alice_and_bob_manually_refund_after_funds_locked_both_refund' panicked at 'called `Result::unwrap()` on an `Err` value: All retry attempts for creating a wallet exhausted

My best guess is that monerod is actually not running correctly?

da-kami commented 3 years ago

Just noticed that my fix of the fix actually never merged - no wonder it is still failing... https://github.com/comit-network/xmr-btc-swap/pull/544 Let's see if that fix is more resilient!

da-kami commented 3 years ago

It appears the previous problem is fixed, but we run into other problems occasionally: 

Run cargo test --package swap --all-features --test happy_path_restart_alice_after_xmr_locked -- --nocapture
   Compiling monero-rpc v0.1.0 (/home/runner/work/xmr-btc-swap/xmr-btc-swap/monero-rpc)
   Compiling swap v0.7.0 (/home/runner/work/xmr-btc-swap/xmr-btc-swap/swap)
   Compiling monero-harness v0.1.0 (/home/runner/work/xmr-btc-swap/xmr-btc-swap/monero-harness)
    Finished test [unoptimized + debuginfo] target(s) in 1m 38s
     Running tests/happy_path_restart_alice_after_xmr_locked.rs (target/debug/deps/happy_path_restart_alice_after_xmr_locked-69a2a4f3b5df923c)

running 1 test
Jun 25 03:21:23.329  INFO testcontainers::core::wait_for_message: Found message after comparing 78 lines    
Jun 25 03:21:28.761  INFO testcontainers::core::wait_for_message: Found message after comparing 2 lines    
Jun 25 03:21:28.761  INFO monero_harness: Starting monerod: CFie_monerod
Jun 25 03:21:36.595  INFO testcontainers::core::wait_for_message: Found message after comparing 183 lines    
Jun 25 03:21:36.652  INFO monero_harness: Starting miner wallet: miner
Jun 25 03:21:38.382  INFO testcontainers::core::wait_for_message: Found message after comparing 16 lines    
thread 'given_alice_restarts_after_xmr_is_locked_resume_swap' panicked at 'called `Result::unwrap()` on an `Err` value: error sending request for url (http://127.0.0.1:49171/json_rpc): connection error: Connection reset by peer (os error 104)

Caused by:
    0: error sending request for url (http://127.0.0.1:49171/json_rpc): connection error: Connection reset by peer (os error 104)
    1: connection error: Connection reset by peer (os error 104)
    2: Connection reset by peer (os error 104)', swap/tests/harness/mod.rs:157:14
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
test given_alice_restarts_after_xmr_is_locked_resume_swap ... FAILED

failures:

failures:
    given_alice_restarts_after_xmr_is_locked_resume_swap

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 19.22s

error: test failed, to rerun pass '-p swap --test happy_path_restart_alice_after_xmr_locked'
Error: Process completed with exit code 101.

It might be that this is related to the fix merged in #544 - potentially the problem is not the wallet RPC but the moneroc container being not available any more because it e.g. crashed.

See run: https://github.com/comit-network/xmr-btc-swap/runs/2910759540