cardano-foundation / cardano-wallet

HTTP server & command-line for managing UTxOs and HD wallets in Cardano.
Apache License 2.0
764 stars 212 forks source link

Flaky test: Lots of integration tests errors due to cluster or setup #3123

Open Anviking opened 2 years ago

Anviking commented 2 years ago

Please ensure:

Context

Some recent failures appear related to the integration test setup, rather than an individual test. It's not just one test failing, but lots of them.

There seem to exist both failures where just stake-pool tests are failing, as well as failures where all tests past a certain point starts failing.

Job name

cardano-wallet:integration

Test case name(s)

No response

Error message

src/Test/Integration/Framework/DSL.hs:1079:11:
  1) API Specifications, SHELLEY_STAKE_POOLS, STAKE_POOLS_JOIN_01rewards - Can join a pool, earn rewards and collect them
       Quantity 0 does not satisfy (> Quantity 0)
       While verifying value:
         ( Status
             { statusCode = 200
             , statusMessage = "OK"
             }
         , Right
             ( ApiWallet
                 { id = ApiT
                     ( WalletId
                         { getWalletId = 08 a5be01a57642239a3c7bd27799c555fd297358 }
                     )
                 , addressPoolGap = ApiT
                     ( AddressPoolGap
                         { getAddressPoolGap = 20 }
                     )
                 , balance = ApiWalletBalance
                     { available = Quantity 999998854600
                     , total = Quantity 999998854600
                     , reward = Quantity 0
                     }
                 , assets = ApiWalletAssetsBalance
                     { available = ApiT
                         ( TokenMap
                             ( fromList [] )
                         )
                     , total = ApiT
                         ( TokenMap
                             ( fromList [] )
                         )
                     }
                 , delegation = ApiWalletDelegation
                     { active = ApiWalletDelegationNext
                         { status = Delegating
                         , target = Just
                             ( ApiT
                                 ( PoolId
                                     { getPoolId = "ì(ó=ËæÖ@
                                     \x1e^3\x9bÐd|\x9sÊl\xcù»æ\x83\x8dÆ" }
                                 )
                             )
                         , changesAt = Nothing
                         }
                     , next = []
                     }
                 , name = ApiT
                     ( WalletName
                         { getWalletName = "Faucet Wallet" }
                     )
                 , passphrase = Just
                     ( ApiWalletPassphraseInfo 2022 - 01 - 27 19 : 40 : 49.004227 UTC )
                 , state = ApiT
                     ( Syncing
                         ( Quantity
                             ( Percentage
                                 ( 9279 % 10000 )
                             )
                         )
                     )
                 , tip = ApiBlockReference
                     { absoluteSlotNumber = ApiT
                         ( SlotNo 3585 )
                     , slotId = ApiSlotId
                         { epochNumber = ApiT
                             ( EpochNo
                                 { unEpochNo = 35 }
                             )
                         , slotNumber = ApiT
                             ( SlotInEpoch
                                 { unSlotInEpoch = 85 }
                             )
                         }
                     , time = 2022 - 01 - 27 19 : 41 : 56 UTC
                     , block = ApiBlockInfo
                         ( Quantity 1572 )
                     }
                 }
             )
         )

       Waited longer than 90s to resolve action: "Wallet gets rewards".

A lot of stake pool related tests were failing, indicating something ended up wrong with the setup of one pool (have not looked close)

Build link

https://hydra.iohk.io/build/12215190

Anviking commented 2 years ago

Seems similar to #2720, but may or may not be a different problem this time.