Functional test randomly fails

ealefad commented 5 years ago

Sometimes a functional test that basically works fine, fails in Travis or local build, e.g. rpc_fundrawtransaction.py fails approximately in 8% cases (8 out of 100 runs)

ealefad commented 5 years ago

Running all the tests 100 times:

feature_config_args                 Fail =   0, Pass = 100, Fail Ratio = 0.00 %
feature_csv_activation              Fail =   0, Pass = 100, Fail Ratio = 0.00 %
feature_dersig                      Fail =   0, Pass = 100, Fail Ratio = 0.00 %
feature_help                        Fail =   0, Pass = 100, Fail Ratio = 0.00 %
feature_logging                     Fail =   0, Pass = 100, Fail Ratio = 0.00 %
feature_minchainwork                Fail =   0, Pass = 100, Fail Ratio = 0.00 %
feature_nulldummy                   Fail =   0, Pass = 100, Fail Ratio = 0.00 %
feature_proxy                       Fail =   0, Pass = 100, Fail Ratio = 0.00 %
feature_reindex                     Fail =   0, Pass = 100, Fail Ratio = 0.00 %
feature_uacomment                   Fail =   0, Pass = 100, Fail Ratio = 0.00 %
feature_versionbits_warning         Fail =   0, Pass = 100, Fail Ratio = 0.00 %
interface_bitcoin_cli               Fail =   0, Pass = 100, Fail Ratio = 0.00 %
interface_http                      Fail =   0, Pass = 100, Fail Ratio = 0.00 %
interface_rest                      Fail =   0, Pass = 100, Fail Ratio = 0.00 %
mempool_limit                       Fail =   0, Pass = 100, Fail Ratio = 0.00 %
mempool_persist                     Fail =   0, Pass = 100, Fail Ratio = 0.00 %
mempool_reorg                       Fail =   0, Pass = 100, Fail Ratio = 0.00 %
mempool_resurrect                   Fail =   0, Pass = 100, Fail Ratio = 0.00 %
mempool_spend_coinbase              Fail =   0, Pass = 100, Fail Ratio = 0.00 %
mining_basic                        Fail =   0, Pass = 100, Fail Ratio = 0.00 %
mining_prioritisetransaction        Fail =   0, Pass = 100, Fail Ratio = 0.00 %
p2p_compactblocks                   Fail =   0, Pass = 100, Fail Ratio = 0.00 %
p2p_disconnect_ban                  Fail =   0, Pass = 100, Fail Ratio = 0.00 %
p2p_fingerprint                     Fail =   0, Pass = 100, Fail Ratio = 0.00 %
p2p_invalid_block                   Fail =   0, Pass = 100, Fail Ratio = 0.00 %
p2p_invalid_tx                      Fail =   0, Pass = 100, Fail Ratio = 0.00 %
p2p_mempool                         Fail =   0, Pass = 100, Fail Ratio = 0.00 %
p2p_node_network_limited            Fail =   0, Pass = 100, Fail Ratio = 0.00 %
p2p_sendheaders                     Fail =   0, Pass = 100, Fail Ratio = 0.00 %
p2p_unrequested_blocks              Fail =   0, Pass = 100, Fail Ratio = 0.00 %
rpc_decodescript                    Fail =   0, Pass = 100, Fail Ratio = 0.00 %
rpc_deprecated                      Fail =   0, Pass = 100, Fail Ratio = 0.00 %
rpc_fundrawtransaction              Fail =   9, Pass =  91, Fail Ratio = 9.00 %
rpc_getchaintips                    Fail =   0, Pass = 100, Fail Ratio = 0.00 %
rpc_named_arguments                 Fail =   0, Pass = 100, Fail Ratio = 0.00 %
rpc_net                             Fail =   0, Pass = 100, Fail Ratio = 0.00 %
rpc_preciousblock                   Fail =   0, Pass = 100, Fail Ratio = 0.00 %
rpc_rawtransaction                  Fail =   0, Pass = 100, Fail Ratio = 0.00 %
rpc_signmessage                     Fail =   0, Pass = 100, Fail Ratio = 0.00 %
rpc_txoutproof                      Fail =   0, Pass = 100, Fail Ratio = 0.00 %
rpc_uptime                          Fail =   0, Pass = 100, Fail Ratio = 0.00 %
rpc_users                           Fail =   0, Pass = 100, Fail Ratio = 0.00 %
wallet_abandonconflict              Fail =   0, Pass = 100, Fail Ratio = 0.00 %
wallet_accounts                     Fail =   0, Pass = 100, Fail Ratio = 0.00 %
wallet_address_types                Fail =   0, Pass = 100, Fail Ratio = 0.00 %
wallet_backup                       Fail =   0, Pass = 100, Fail Ratio = 0.00 %
wallet_bumpfee                      Fail =   0, Pass = 100, Fail Ratio = 0.00 %
wallet_disable                      Fail =   0, Pass = 100, Fail Ratio = 0.00 %
wallet_dump                         Fail =   0, Pass = 100, Fail Ratio = 0.00 %
wallet_encryption                   Fail =   2, Pass =  98, Fail Ratio = 2.00 %
wallet_hd                           Fail =   0, Pass = 100, Fail Ratio = 0.00 %
wallet_import_rescan                Fail =   0, Pass = 100, Fail Ratio = 0.00 %
wallet_importmulti                  Fail =   0, Pass = 100, Fail Ratio = 0.00 %
wallet_importprunedfunds            Fail =   0, Pass = 100, Fail Ratio = 0.00 %
wallet_keypool                      Fail =   0, Pass = 100, Fail Ratio = 0.00 %
wallet_keypool_topup                Fail =   0, Pass = 100, Fail Ratio = 0.00 %
wallet_listreceivedby               Fail =   0, Pass = 100, Fail Ratio = 0.00 %
wallet_listsinceblock               Fail =   3, Pass =  97, Fail Ratio = 3.00 %
wallet_multiwallet                  Fail =   0, Pass = 100, Fail Ratio = 0.00 %
wallet_resendwallettransactions     Fail =   0, Pass = 100, Fail Ratio = 0.00 %
wallet_txn_clone                    Fail =   0, Pass = 100, Fail Ratio = 0.00 %
wallet_txn_doublespend              Fail =   2, Pass =  98, Fail Ratio = 2.00 %
wallet_zapwallettxes                Fail =   0, Pass = 100, Fail Ratio = 0.00 %

ealefad commented 5 years ago

some Bitcoin tests display the same:

p2p_unrequested_blocks              Fail =   0, Pass = 100, Fail Ratio = 0.00 %
rpc_fundrawtransaction              Fail =   0, Pass = 100, Fail Ratio = 0.00 %
wallet_encryption                   Fail =   4, Pass =  96, Fail Ratio = 4.00 %
wallet_txn_doublespend              Fail =   0, Pass = 100, Fail Ratio = 0.00 %

Equibit / equibit

Functional test randomly fails #330