Mempool out of sync - Githubissues

blockparty-sh commented 4 years ago

Issue occurred with this tx:

1e6442a0d3548bb4f917721184ac1cb163ddf324e2c09f55c46ff0ba521cb89f

cpacia commented 4 years ago

What version of bchd are you using?

blockparty-sh commented 4 years ago

What version of bchd are you using?

0.16.5

readcash commented 4 years ago

Unfortunately this keeps happening. BCHD happily reports that transaction is accepted, doesn't broadcast it further, responds as if transaction exists...

Happened today with 858f20eeffb834ee6ba88a09d6b4fcb0b0d379d5efcd0a0bddca65e6c6153513 (nowhere to be found), also quite a few daily reports from people who send money to someone and then later (after a restart) the money disappears. Happens daily.

How can we find out the root cause?

These transactions were affected (we have the hexes, so that means that BCHD accepted them):

858f20eeffb834ee6ba88a09d6b4fcb0b0d379d5efcd0a0bddca65e6c6153513 e6905b9ecfe88f6b654f70696a16534200ecde76253d929b19c67c08570dadc9 840a21e7df5ba913ec841ec02c2ed78b53f9db2f16a70dbd99ba6374b58b1a74 4be547f079b3801c8eef09f9aa0c278ca6fcdfd5887e820fb3ccf149fbf9639e

Unfortunately, it seems that BCHD has already restarted, so I can't find the information about them anymore.

We can we do? (My mailbox is now full of people (these transactions were to ~400 people) panicking that their money was stolen - there's a notification that they got the money, but the money is now gone... after BCHD restart, when mempool was reset)

readcash commented 4 years ago

The fee for tx was calculated by BITBOX this.BITBOX.BitcoinCash.getByteCount then multiplied by 1.01, then Math.floor.

Size: 3,625 Fee: 0.00003662 BCH. So the failed transactions probably had a similar fee. So, it doesn't seem to be a fee issue.

It doesn't only happen with big transactions - sometimes transactions with just a few inputs-outputs fail. (By "fail" I mean that they are accepted by BCHD, appear in mempool, can be spent, not broadcasted to the network or maybe the network doesn't accept the broadcast, disappear upon node restart)

Is there a way to enable verbose logging in BCHD maybe?

readcash commented 4 years ago

Even more failed/non-relayed transactions today: ec750e7ebc80ce7f4a8b4ffe770e3e0161aeb9608fcd69e79ec764867714271d 922980460d8cca31ddea3b73b7b6593d9b1b2534d0a35c6af3709e148934dece cfa0a31369eb1a53ebca71233f0dde2de0a100d2c7f74e6d5164284ac1ca8b1f e93a1fb2a9b76d980fb464b6ae672c02890dd52fde8b48656ddf02148ce7741f 8b450af44424f1a26ce64075a65983cca413b95b9444afdbbf60ea840de7ed50 8af074e73890810d39a072716c0f92a594eb225c19cf933072e99ec7c2cd07eb ...

Hundreds of people affected. Email blown away. What can we do?

readcash commented 4 years ago

Finally, caught one in the act:

0200000001337745929a9f8ac6726ef83b2e17e2def07550013d2c1efe70449f536a5fdb93650000006a47304402206f3c59b2778c8e417cb2c01b9df5d62f3e8583b83f341beddf3ffe5f180fe80802204124333993585b21c75b6db8b27562e5191418263114374a02ad90de5ba8b19241210337d4dfb6295828b4e573e69a2e13ec564df92703a8adffa9d0de7c3de45023f4ffffffff66b1c60400000000001976a914e38f909fa9c3ad80411c20ee19e5f8d53456cab088ac4e670100000000001976a914f2c0361017eb347fb18674f4e1a20775ff4df88688acdca30000000000001976a9145742c20fa8c850e30232b90e759c97f18dc4afd288ac64610000000000001976a914630b00c58d1973e310d20704c1f5ba4580f721bc88ac03320000000000001976a914876ca724a7b5f4ca4512786dc4cb2a4141a20d6888ace99f0000000000001976a914459edd5d4554ead9df18c8583290bbe3091a59af88ac96850000000000001976a9144520c7d04669eb79cfd0ec3f46a2d1125744ded788acea520000000000001976a914093038c983ce5b9afb8c1df20b79b02113a41c9088ac43050000000000001976a914d71059e76cfb7a1407b4c74ad6d340ff4c7323df88ac725d0000000000001976a9141bbec56ead11114707fedb19bce646637cf2f05d88ac82250000000000001976a914a3e859de5e7ca1de4fb56270f92b5c919268af5f88ac1b900100000000001976a914aefd1d2672cd949b97f4b6eb163899fdaa59f58188acda8a0000000000001976a914932be05800d8d2b7e4a06017a18f65ac85a0c10088ac43050000000000001976a9142039dc54335d4ce64921cabd70459641e766b43688acf8800100000000001976a91489823fdee8db38663337cac9dbb3aa36c87ce4b388ac93850100000000001976a914f9ad445719e34f347a3d17c128eb68af617fe23588acfb800000000000001976a91444379aeb2d81b5d477af426e7b29221bbcc1229188ace7520100000000001976a914d74634e58855e8cb8c4ea15a23f500f7134b626288ace2a10000000000001976a914b87478c7a1a5f78584a86a966c2973042c012a7f88acdf6f0000000000001976a914d108f0a239633ea6a87173124903c9ae170fc61288acc7430000000000001976a91424d5590859dba099b0a18660815c1dfe5fc12cee88ac7a0e0000000000001976a9146e05fa573ab162c1695c0ff06cf76c4caeb61df788ac6f5d0100000000001976a91443562b911a8e105b9591b6f36083f1c04f7bcd4788acb5620000000000001976a9141c932fc392802cbc859c1490fc5f45dbf20202b288ac75f50000000000001976a914d37fc0f50c981846763120b9257f89325f18593d88acf41c0000000000001976a9142ba3259f86c3bbb824d29ef0d653c3876449137588ac77740100000000001976a914c25bfc47cb5734a5c199f8f025e0e5d1356b490688ac13450200000000001976a9140ca81ee972a39d43df187495c542282e312650b088ac6bc50000000000001976a91408e380abb0dc11ef40d70b1ee783949eb6101c0e88accbdb0000000000001976a9146e433f618950abd774ab63d7000f4ad5446ad7fb88ac90b90100000000001976a914d42f293ff956194da5a11dcf58917d17ce4594fe88ac94060000000000001976a9147b8fa6a38fb2bc18254e525e79434b24f24663f488ace1ba0100000000001976a9149b24a82d959366479c84a85d83a4eae920a6d3df88ac17920000000000001976a91451716b67a7b78c81f362c2bad3e7df274f98d4fc88aca3810000000000001976a91459b625f366daacdd49ed87080f930a9645ff01bc88ac08e30000000000001976a91474b721d1f368ec0631a1efeabfbcd07ead7d548b88ac38390300000000001976a9140c29d20311f3a034330bb4d427731ecac4590cec88ac82250000000000001976a9140a3664feaa59d826d4afd32103e44f93012a0f7888acf7cd0100000000001976a9140056ddfa3420b747ff4b4b123a661c5e575d7b0488acae7d0100000000001976a9144b5c9869c04f55af929a2bb5f6d72e54edebde1688ac333d0000000000001976a914e9eec2a3f0ddd91e5ec7a59170d0b02d49a9c0cb88ac8c550000000000001976a914c2ffb2a2a1bd32355635a0338873d9ea7219ce1888acc1900200000000001976a91467509fcec36b35613226132ebcb1fddc500e538588ac94060000000000001976a91440c1b19cfe5ed7626587399b7bcd6cebcbf1b53288ac55650100000000001976a9142476fec86f0010077c9269d61e6e82d6414569d888ac59e40000000000001976a9149be4a2d0d738763258da5f02917a7d6289632a5588ac08490100000000001976a9140d575a9e45eedcb5b79da8b474b88e6829c478d988ac785b0000000000001976a91421b48743f183ffab1634d1a3f7c01d82e17888e088ace6050200000000001976a914ec623ff577ad5768a36f18e2e618043fd4f7170788ac3bd10300000000001976a91496335632b86d7004976dba3c06bd2e86502f503b88ac29d90000000000001976a9149bc097e292265aa4236058a658530e2aff91c8af88ac3e200000000000001976a9141982b45bac0bd189726c7f9478e9d58ae3c8716a88aced9d0100000000001976a914524d968b5c44852dd6f7f81fc6da3ad9fe34908e88ac73760000000000001976a914942a94804f6a58a7043f4f7a70c48564ad3bcbbb88ac47370000000000001976a9143eddc77879aac6fd323b01d6011dec72c5e640c088acf3cf0000000000001976a9141a9b336d5b079b38ae66a7a74ffd4286353f8fdf88acaa190000000000001976a9143d25840b7a192afd2c973dfd8866e2efe2c6d47d88ac607c0000000000001976a9148cb2a6ec343982ecb4127814331d5f5bf87b97cf88acb2e10200000000001976a9149ef478476401496d2cb901283a17fcf529e3878688ac36ba0200000000001976a9140022e64a91133012bea5308768c705da61b268d288ac56180100000000001976a91429b3f06eebffacb658727cfb2b5a5e7c1bcc8b2f88ac85d60100000000001976a9149ac74895b308ce7d6e3c818919926b14a5ef136a88ac2e090300000000001976a9143f2309e1e80225f170a87ae1205464c27d9d754488ac08e30000000000001976a9142413e5b9a1f99d1dae881be65d98bfd80b13afe188ac96850000000000001976a914991c477b8c459318fe9a5b43024b1789afff250688ace5070000000000001976a914ccdc9ef9c3e27e5fc13acdf3c27575c4da90847088ac5dae0200000000001976a914d6f82bcbc3cff9927e65c2e9598e338a11b9f82388acaae50000000000001976a914d5d2c29c6ccf36110edce55bfec0973f7102204188ac798d0100000000001976a9141641386c4a269c8de608dc6fa0027ceb00ab8e2288acaf960100000000001976a9149a23a63ca64d1e225c914a4a249fa6e680d88e3c88ac76f30200000000001976a9140a38310880d72e1f16cdedb2ec6d548a5d56626888acb7e10000000000001976a914e825f649e25a30a1e64af3f35ac16ac75dbd521188ac4e010100000000001976a91427667104c3dbdcaf7c3a941454a1d35477f90b9688ac76270200000000001976a914d3feffad1ecfe8a5641e8a53e26f82d6d73a482c88acf39b0100000000001976a9142eafea23ea42bb03fcb8e7b0e664427a5b9ab12988ac2eb40c00000000001976a914a56928931e7e34c6f8f6f0bde83172507cfc279f88ac16f60300000000001976a91495c7cdf99bacaaa166d673364f1588cd6b11392088acf4820000000000001976a914ecd82ec6b193d4eaa9128a4843493b7e7c9b62de88aca01b0100000000001976a91435c19920907a8656953a02740d999e468590243688acde090000000000001976a9145e52008f0a39c0f82e4de95f3bd24f0318eb21d388ac411c0400000000001976a914e4b7943d6f960c41314174d4af6b8f3adafc530988ac08170000000000001976a91448a777ef72322229e5a40177c63366d2d32f2b3788acb5620000000000001976a91403b05270274c16a354703a37966481b28a2e96a888ac3e200000000000001976a9144222142b9770c5e0eb2da38da60a7a2779a108e688ac26410000000000001976a9141da7f4cd1e72cf5a2406fc7ce6fb385241bdbfb888ac95040200000000001976a9143655490b1952841a75ab0bd90287f9e83307345488ac2a0b0200000000001976a914d91ced14fc1790a8f07b35c5fbab3d1978b89b0888ac870a0000000000001976a914df715984baa687647f1f36719398c6b7a3ec740988ac225a0200000000001976a9141c883dbacc389e6de86244d6c69678a6ed5c384988ac21a50400000000001976a914c9bf7dd111562abebcbb78b920bd9ecc07a83dc788ac612f0000000000001976a9146b550b57db16e61149c6f4297dfc0b1859cf0f9388ac7da20500000000001976a91455465b51b1d44ae61ae1ebc5cec476031920522988acef370000000000001976a914383b7baffb9741e4d1e8699b84c77a029275cb0d88ac70440000000000001976a914f91e412ab265aa8ce81ca6ff192ce4eb655ecf5688ac5f950100000000001976a914f19e55623085e3a114e4263fd068e91ddf547a6588ac9bb50200000000001976a9145ab71704b19d4b7156e55b5cd27f82ea052e297188acd2730000000000001976a9148dae6fd3ea4ca150e2aa3e551458158069aef5d288ac36090000000000001976a914e5e16943d378d83f3f6b6600baf9cbb409a30f1288ac8e3a0100000000001976a9141d410b9eea9a4fad0049235b7f4c6fa9b2ae25e988ac255a0100000000001976a914b2f750f6f456ee90df09b47083d5f8a7b99451bf88acf3fd0600000000001976a91497073bb85895dca6e6810783515558aceed8cdaf88ac5d72ed0a000000001976a9140310146cd1252fabe79848e23fc89eb5dbd2519688ac00000000

When we first sent this transactions (via GRPC SubmitTransaction), we've got back this:

2f2f022d3579356a20ec985e4bdb9d30f8982b95cf2ca4e82771b71d454f6aca

So, the transaction was accepted.

If we go to https://explorer.bitcoin.com/bch/search/2f2f022d3579356a20ec985e4bdb9d30f8982b95cf2ca4e82771b71d454f6aca

We can see that the transaction did not reach the network:

If I try to SubmitTransaction again (though this time via Swagger over GRPC), I get:

{"code":3,"message":"tx rejected: already have transaction 2f2f022d3579356a20ec985e4bdb9d30f8982b95cf2ca4e82771b71d454f6aca"}

So, this transaction is in the mempool, but not broadcasted.

Finally, I went to https://rest.bitcoin.com/#/rawtransactions/sendRawTransactionSingle

I get:

{
  "error": "Missing inputs"
}

Ok, let's decode and see the inputs

"n": 101,
"txid": "93db5f6a539f4470fe1e2c3d015075f0dee2172e3bf86e72c68a9f9a92457733",

Here's the 93db...:

0200000001f48da90952a82cf0ab54dc70fb86cbfb61b4783979cf2f38575d54985b8c59d6650000006a473044022062364faa1f790d459d10cb6d9fbf3b22d51fad0ed296bcd31e10c66b38cc013602203c124cf7daac3c1bc691d653223c1be1f76edef6f25b1667967291fd9ae33fb941210337d4dfb6295828b4e573e69a2e13ec564df92703a8adffa9d0de7c3de45023f4ffffffff66d2730000000000001976a914c80913c1bc7b192313e8dc540adf0d26327020d088acdfd50000000000001976a91431e12e9e4f81c4ae23d3fd2db59e46b96a3b575b88ac4d350000000000001976a91448095c20865cf54c3ded5e5b19456209ad853ddf88ac0eab0400000000001976a91424a5275547c1c2969d5b1ecd8ea60b1124139b4288acf9cd0000000000001976a91443b0ebf9d4c536241c0dbbf796ef26595757c71688ac2b260000000000001976a914296cf08efbf713f2f7d79fa0ccf6b71f99f2d17488ac4a030000000000001976a91448ffcbecc756fa449b7be2d6ec4c14bb7b45303788acc5260500000000001976a914d0f7c9cdfbedb336878b6ee73a9e81add984aee888ac7c8d0000000000001976a914eb447668b2c48c57d4bd634b0b12cfe5684d097488ac054b0000000000001976a914e4da71d713e2cd0631b65dd4b05512dfb9b97fd288ac16770200000000001976a914fed734a282c63a27ac8ff7f0722f194fc98b022d88ac4fb40000000000001976a914a3feae822f876fdd00d5b6dff229774e858b34a388acfb4c0100000000001976a914ad8f760cd173ece26c5194c694cd13cc69d0740188ac1b5c0200000000001976a9143e97120b91f85a8b7f7b3973f204e2ba9c3b797688ac47370000000000001976a91422622ce29ba24e9669b50296b9a014082b770d6588ac63480000000000001976a9141ac8a0728bfae5a79f868dd251faaa83abecedd188acbd130000000000001976a914194917fbd49ba3a8d896fb51da315657c25cc78088aca1020000000000001976a914f486d550fbf4290ceb3362d5acad8b19278d83ee88ac1c110000000000001976a9148ceec7a1010f5f388dce2abbf01cd05560a663d788ac7a0e0000000000001976a914e9648545e58606988e77f5dc5176fb9f12c79bae88ac99510000000000001976a91447865fe927dd6e24bf3597fb62f50f2a39801cff88ac2bff0f00000000001976a914472aa11b3eb05a2f4b9f9f0a069810fa050623d888ac333d0000000000001976a9147cfb29b881275f73ed04fd60037fbcb25430d6ed88ac102e0000000000001976a914acd4a828d327dbbc1ffbef3c676bdb08ce9975b188ac5c950200000000001976a914e4dd55614dae5283b81ed72b1c5a578a402cfe5d88ac17c00600000000001976a914c502641bdfcb4cf6d3ab7a3f086d45c04e18fb5b88ace4540000000000001976a91462c03fdd56f86dd850311b8027315baf79d48d9d88accfb80a00000000001976a914a06362537aec9e25ea3e31a2e6a082b7ef8358e788ac1c110000000000001976a9144332725f2b854ba9c921a349c9265df82cb2520d88ac220f0000000000001976a914dce77be0512296c0dcf185995793b6c8f1093d0d88acdd560000000000001976a9145b7290fa82a4b589c6270291a722545124491b1d88acc35e0000000000001976a914459eca601d7e804ff217fe83982553f6fd36dd7688ac9e980500000000001976a91466562c47ef1209ebef6969b67cbdf515c84da7fa88acd4be0100000000001976a91436495196d3a9263dc87f49c4b21a83a36743280d88ac77420000000000001976a9140e2719669c926ba2a2e0ce3b827c4e68f33f77cf88acdfb80400000000001976a91402646e4dc9d3f951a5ce995d4d065ba98b57f21988accd8e0000000000001976a914a243df3d86a04a901c8048c9f2ceb5f7804ef5e288acfcc90400000000001976a9143d8a4cc5b08549d46c8d8a427115d9cfba9de62e88ac77420000000000001976a91465d7a31fe5a5187ee4f24c9274e3cedc878a3c4c88acdf6f0000000000001976a914d2a0176564703e487065855d31a86f29be49de9e88ac2e3b0400000000001976a914dea1f8c931ad35091e88a0b53dc07ea52378b74188acd8050700000000001976a914bb6ab264a6ccf1e20318983979d3bd9d37ec47af88ac70440000000000001976a914655b06b914f84a9d663a17c8c35858b672f6e4b888ac2b8c0000000000001976a9147bdf2d643dc699ca202aed72a667711561b32b2888acfb1a0000000000001976a914a960597a599345b4738c4252fb78a017cf9d9d1088ac9ee70200000000001976a914420dbbe0a2e74074475d34704f3727cb5d56661188ac44d10000000000001976a914b3b0f6f8464c36f40d52396403f3f3083948bb7e88ac48500000000000001976a9148b1f549938e49da41e1cb8dcde470b2d3c2127f988ac800c0000000000001976a9149ceaa2be46c257942e0ad5866dbb234959f1f4a888acb7150000000000001976a914841174af13c78c02a671d876dbd4fc2fe33ca73a88aced1e0000000000001976a9140fc11b38181f000ba31ab73431aa80195bf8016788ac01190000000000001976a91494111523e18857416e7b86e9e9777b5831869ad888ace7200000000000001976a914cb3bb098dafa22f7ba8f683083d2f87a7a5b55a388ac81720000000000001976a914f98b534af5d4ee6fe6dff0143ef72e17df1ffb9788ac26410000000000001976a914459f62b720b51507ed780b29cb67d5e382d150de88ac612f0000000000001976a9143e078be84ce4c90b53997743164d90cde35f9cd788acd20b0200000000001976a914df4aac5a1ae60f8ad51eb1bd4df8c1aecb10f58e88acb77b0000000000001976a9149977abf0a1684c669f7f66d6e0b7cacb9db1d69188ac85570000000000001976a9147ef244e0f18b87d1a9045cf9b947a751c6af17ba88ac7d400000000000001976a914d855c1ef0456be8bacad229051feb98297805b5e88acfb4c0100000000001976a914a2cd6bd411310e7c8b89efa64298a0279c79ec6488ac91a00000000000001976a9142de267f3d50ff5015fec1882d93ed978f36f602b88acb2300000000000001976a914e59b7f4fc9f6dbc85a312fdb68683166d96df10188ace7d10200000000001976a9142762ecc93030c89f1e8453650feb4348a674377188ac17920000000000001976a9143e7bf680d92ec7aaac5ecbe79ac5900e2908314b88accd8e0000000000001976a914aff62371afea0fd33cddc0dfcfdb9510eb71320a88ac5a310000000000001976a914686631ab2777550c7bd3032c8654d56c0eb6f70c88ac31240000000000001976a914096d48f0080b1ffea8c189392ee16a560312e81888ac71f30400000000001976a914d61baaf3beac44b7028f3fde449f80fb12a6484f88ac142c0100000000001976a914c0ea5ccb351ed344662f414bd6e3fb47f016396288ac84210400000000001976a914178fd2504f0cceebfb0fd809222c559f565c747288ac6e910000000000001976a914eee133dc47ab88fc982115753ffcff49e1191b4e88ac66420600000000001976a91427005d2d21003fd70af118a6d99986a34d35c61a88ac51670000000000001976a914027042fa428dc63d48fd6d331218cab36dc0618d88ac521a0000000000001976a91477d722f297b81ed2a5b9dcc595b881dd6bb470c788ac3aa10000000000001976a914e00564e483a88970166db200c00fcda2cec09f8d88ac73100000000000001976a91418d025ed9a8baf41ce0915b9cc65d9395394518d88acbf2c0000000000001976a914a36d14b118df4a1370f744a41560656744dbf23f88ac5b630100000000001976a9141768753b454d971636f560cefcf710a092005b9988acf9cd0000000000001976a914ed93f41278f187e2d8cdef97f771bdb91846706188ac828b0000000000001976a91450fe7c1c4537511fe61d90f0a479a845738042dd88ace5070000000000001976a91400b570df9d4eb0c18f5158df832de75caa3731c688ac36090000000000001976a9149bd05df344a49a758ec4624d44f5a1dedab5f48288ac33090100000000001976a914e7b522baf4032dd0cd28c2195f7de6be2346374c88ac273f0200000000001976a914d4abe05dd073e85df42f3e7be2811819d8c962f988ac71dc0100000000001976a9144240788141f01113b278922dc962721dac7c93aa88acf2030000000000001976a9143bde6364e7459f10d7242adcf785a7a3f898e76388acc7430000000000001976a914566ccfff1041949cbb1c79a2c603b9ddc281a4f188ac01190000000000001976a91478a28e906f3435dbbdd38d428581a77948eeb5e688acb9940000000000001976a91450d4c73d5d588bad86260daea6f04437554b4d8488ac07ca0000000000001976a914631ea3ab31f0066c346a0781f3f6a6cf4316aed988ac6e910000000000001976a914fa0be6f9fcd8df4dcc6e96ad300a67c3645629e488ac4d9b0000000000001976a91476369367e26acd20649d5727fd520639b500351d88acb5620000000000001976a9146be2654d3642e136bb9ea6149ed8a2e641d5729388acd10d0000000000001976a914af7ca948770bedbbf4c6ae9f0cd7f4f57a03bcef88ac0a300000000000001976a9149bb32a0db659512c6be4da54fd3d6792a50896e288ac479d0000000000001976a914e802b866eecaa95e49e95da8c55ae6a56fecd47e88ac172c0000000000001976a914a975a305b144f67fe6ac3c9cafd80192fc1ed10488ace0220000000000001976a91439e1092a08257cec01a538826e39b500dcccaf3e88ac26410000000000001976a9144b000c344c5b3179cb1bdc74beaf7d07884451d388acf3cf0000000000001976a914acb4ab0b688d9a477f9355c0db5d8a711f5e9dee88ac228c780b000000001976a9140310146cd1252fabe79848e23fc89eb5dbd2519688ac00000000

If I submit it to BCHD I get:

tx rejected: already have transaction 93db5f6a539f4470fe1e2c3d015075f0dee2172e3bf86e72c68a9f9a92457733

If I submit it via rest.bitcoin - I get:

{
  "error": "txn-mempool-conflict (code 18)"
}

Now, I'm kind of stuck. I found this: https://updates.cryptoapis.io/?lng=en&page=5

Many customers got this error using our APIs or with their own node. It happens if you try to broadcast a second transaction and the first one is still not mined, which means it doesn't allow you to create more than one transaction from same address in the same block.

But this doesn't really help. A double spend? How?

The parent of this transaction is:

d6598c5b98545d57382fcf793978b461fbcb86fb70dc54abf02ca85209a98df4

and indeed, we can see this transacation on BCH network and we can see that 8d6da5c6fc593944a7c814c250fd49c03b05a756951002a37eae7887b6de6436 spends the last output.

The order of sending was:

get utxos 8d6da5c6fc593944a7c814c250fd49c03b05a756951002a37eae7887b6de6436 (broadcasted OK) get utxos e248b666723ba4b63a02a604d95a376d54b44e5ebea3c376ffc1d5615c3b8c03 (broadcasted OK) get utxos 3560939bb09447a471302c57ccbe9826a9eb138ca8d8e4c12362fd44453e4bd9 (broadcasted OK) get utxos a1596215808eca3f31269e80667379da6af80426c91cf114a486667ff9974bef (broadcasted OK) get utxos 93db5f6a539f4470fe1e2c3d015075f0dee2172e3bf86e72c68a9f9a92457733 <--- NOT OK (mempool conflict, tries to spend d6598c5b98545d57382fcf793978b461fbcb86fb70dc54abf02ca85209a98df4)

This means that "get utxos" (GRPC call to the same node) got us d6598c5b98545d57382fcf793978b461fbcb86fb70dc54abf02ca85209a98df4 (which was long before that and was already spent by 8d6da5c6fc593944a7c814c250fd49c03b05a756951002a37eae7887b6de6436 - this is the first transaction we sent in this batch!)

How is this possible?

readcash commented 4 years ago

To be clear: 93db5f6a539f4470fe1e2c3d015075f0dee2172e3bf86e72c68a9f9a92457733 (spending of spent UTXO) was accepted by BCHD, but never left the mempool.

readcash commented 4 years ago

Hex of 93db5f6a539f4470fe1e2c3d015075f0dee2172e3bf86e72c68a9f9a92457733:

0200000001f48da90952a82cf0ab54dc70fb86cbfb61b4783979cf2f38575d54985b8c59d6650000006a473044022062364faa1f790d459d10cb6d9fbf3b22d51fad0ed296bcd31e10c66b38cc013602203c124cf7daac3c1bc691d653223c1be1f76edef6f25b1667967291fd9ae33fb941210337d4dfb6295828b4e573e69a2e13ec564df92703a8adffa9d0de7c3de45023f4ffffffff66d2730000000000001976a914c80913c1bc7b192313e8dc540adf0d26327020d088acdfd50000000000001976a91431e12e9e4f81c4ae23d3fd2db59e46b96a3b575b88ac4d350000000000001976a91448095c20865cf54c3ded5e5b19456209ad853ddf88ac0eab0400000000001976a91424a5275547c1c2969d5b1ecd8ea60b1124139b4288acf9cd0000000000001976a91443b0ebf9d4c536241c0dbbf796ef26595757c71688ac2b260000000000001976a914296cf08efbf713f2f7d79fa0ccf6b71f99f2d17488ac4a030000000000001976a91448ffcbecc756fa449b7be2d6ec4c14bb7b45303788acc5260500000000001976a914d0f7c9cdfbedb336878b6ee73a9e81add984aee888ac7c8d0000000000001976a914eb447668b2c48c57d4bd634b0b12cfe5684d097488ac054b0000000000001976a914e4da71d713e2cd0631b65dd4b05512dfb9b97fd288ac16770200000000001976a914fed734a282c63a27ac8ff7f0722f194fc98b022d88ac4fb40000000000001976a914a3feae822f876fdd00d5b6dff229774e858b34a388acfb4c0100000000001976a914ad8f760cd173ece26c5194c694cd13cc69d0740188ac1b5c0200000000001976a9143e97120b91f85a8b7f7b3973f204e2ba9c3b797688ac47370000000000001976a91422622ce29ba24e9669b50296b9a014082b770d6588ac63480000000000001976a9141ac8a0728bfae5a79f868dd251faaa83abecedd188acbd130000000000001976a914194917fbd49ba3a8d896fb51da315657c25cc78088aca1020000000000001976a914f486d550fbf4290ceb3362d5acad8b19278d83ee88ac1c110000000000001976a9148ceec7a1010f5f388dce2abbf01cd05560a663d788ac7a0e0000000000001976a914e9648545e58606988e77f5dc5176fb9f12c79bae88ac99510000000000001976a91447865fe927dd6e24bf3597fb62f50f2a39801cff88ac2bff0f00000000001976a914472aa11b3eb05a2f4b9f9f0a069810fa050623d888ac333d0000000000001976a9147cfb29b881275f73ed04fd60037fbcb25430d6ed88ac102e0000000000001976a914acd4a828d327dbbc1ffbef3c676bdb08ce9975b188ac5c950200000000001976a914e4dd55614dae5283b81ed72b1c5a578a402cfe5d88ac17c00600000000001976a914c502641bdfcb4cf6d3ab7a3f086d45c04e18fb5b88ace4540000000000001976a91462c03fdd56f86dd850311b8027315baf79d48d9d88accfb80a00000000001976a914a06362537aec9e25ea3e31a2e6a082b7ef8358e788ac1c110000000000001976a9144332725f2b854ba9c921a349c9265df82cb2520d88ac220f0000000000001976a914dce77be0512296c0dcf185995793b6c8f1093d0d88acdd560000000000001976a9145b7290fa82a4b589c6270291a722545124491b1d88acc35e0000000000001976a914459eca601d7e804ff217fe83982553f6fd36dd7688ac9e980500000000001976a91466562c47ef1209ebef6969b67cbdf515c84da7fa88acd4be0100000000001976a91436495196d3a9263dc87f49c4b21a83a36743280d88ac77420000000000001976a9140e2719669c926ba2a2e0ce3b827c4e68f33f77cf88acdfb80400000000001976a91402646e4dc9d3f951a5ce995d4d065ba98b57f21988accd8e0000000000001976a914a243df3d86a04a901c8048c9f2ceb5f7804ef5e288acfcc90400000000001976a9143d8a4cc5b08549d46c8d8a427115d9cfba9de62e88ac77420000000000001976a91465d7a31fe5a5187ee4f24c9274e3cedc878a3c4c88acdf6f0000000000001976a914d2a0176564703e487065855d31a86f29be49de9e88ac2e3b0400000000001976a914dea1f8c931ad35091e88a0b53dc07ea52378b74188acd8050700000000001976a914bb6ab264a6ccf1e20318983979d3bd9d37ec47af88ac70440000000000001976a914655b06b914f84a9d663a17c8c35858b672f6e4b888ac2b8c0000000000001976a9147bdf2d643dc699ca202aed72a667711561b32b2888acfb1a0000000000001976a914a960597a599345b4738c4252fb78a017cf9d9d1088ac9ee70200000000001976a914420dbbe0a2e74074475d34704f3727cb5d56661188ac44d10000000000001976a914b3b0f6f8464c36f40d52396403f3f3083948bb7e88ac48500000000000001976a9148b1f549938e49da41e1cb8dcde470b2d3c2127f988ac800c0000000000001976a9149ceaa2be46c257942e0ad5866dbb234959f1f4a888acb7150000000000001976a914841174af13c78c02a671d876dbd4fc2fe33ca73a88aced1e0000000000001976a9140fc11b38181f000ba31ab73431aa80195bf8016788ac01190000000000001976a91494111523e18857416e7b86e9e9777b5831869ad888ace7200000000000001976a914cb3bb098dafa22f7ba8f683083d2f87a7a5b55a388ac81720000000000001976a914f98b534af5d4ee6fe6dff0143ef72e17df1ffb9788ac26410000000000001976a914459f62b720b51507ed780b29cb67d5e382d150de88ac612f0000000000001976a9143e078be84ce4c90b53997743164d90cde35f9cd788acd20b0200000000001976a914df4aac5a1ae60f8ad51eb1bd4df8c1aecb10f58e88acb77b0000000000001976a9149977abf0a1684c669f7f66d6e0b7cacb9db1d69188ac85570000000000001976a9147ef244e0f18b87d1a9045cf9b947a751c6af17ba88ac7d400000000000001976a914d855c1ef0456be8bacad229051feb98297805b5e88acfb4c0100000000001976a914a2cd6bd411310e7c8b89efa64298a0279c79ec6488ac91a00000000000001976a9142de267f3d50ff5015fec1882d93ed978f36f602b88acb2300000000000001976a914e59b7f4fc9f6dbc85a312fdb68683166d96df10188ace7d10200000000001976a9142762ecc93030c89f1e8453650feb4348a674377188ac17920000000000001976a9143e7bf680d92ec7aaac5ecbe79ac5900e2908314b88accd8e0000000000001976a914aff62371afea0fd33cddc0dfcfdb9510eb71320a88ac5a310000000000001976a914686631ab2777550c7bd3032c8654d56c0eb6f70c88ac31240000000000001976a914096d48f0080b1ffea8c189392ee16a560312e81888ac71f30400000000001976a914d61baaf3beac44b7028f3fde449f80fb12a6484f88ac142c0100000000001976a914c0ea5ccb351ed344662f414bd6e3fb47f016396288ac84210400000000001976a914178fd2504f0cceebfb0fd809222c559f565c747288ac6e910000000000001976a914eee133dc47ab88fc982115753ffcff49e1191b4e88ac66420600000000001976a91427005d2d21003fd70af118a6d99986a34d35c61a88ac51670000000000001976a914027042fa428dc63d48fd6d331218cab36dc0618d88ac521a0000000000001976a91477d722f297b81ed2a5b9dcc595b881dd6bb470c788ac3aa10000000000001976a914e00564e483a88970166db200c00fcda2cec09f8d88ac73100000000000001976a91418d025ed9a8baf41ce0915b9cc65d9395394518d88acbf2c0000000000001976a914a36d14b118df4a1370f744a41560656744dbf23f88ac5b630100000000001976a9141768753b454d971636f560cefcf710a092005b9988acf9cd0000000000001976a914ed93f41278f187e2d8cdef97f771bdb91846706188ac828b0000000000001976a91450fe7c1c4537511fe61d90f0a479a845738042dd88ace5070000000000001976a91400b570df9d4eb0c18f5158df832de75caa3731c688ac36090000000000001976a9149bd05df344a49a758ec4624d44f5a1dedab5f48288ac33090100000000001976a914e7b522baf4032dd0cd28c2195f7de6be2346374c88ac273f0200000000001976a914d4abe05dd073e85df42f3e7be2811819d8c962f988ac71dc0100000000001976a9144240788141f01113b278922dc962721dac7c93aa88acf2030000000000001976a9143bde6364e7459f10d7242adcf785a7a3f898e76388acc7430000000000001976a914566ccfff1041949cbb1c79a2c603b9ddc281a4f188ac01190000000000001976a91478a28e906f3435dbbdd38d428581a77948eeb5e688acb9940000000000001976a91450d4c73d5d588bad86260daea6f04437554b4d8488ac07ca0000000000001976a914631ea3ab31f0066c346a0781f3f6a6cf4316aed988ac6e910000000000001976a914fa0be6f9fcd8df4dcc6e96ad300a67c3645629e488ac4d9b0000000000001976a91476369367e26acd20649d5727fd520639b500351d88acb5620000000000001976a9146be2654d3642e136bb9ea6149ed8a2e641d5729388acd10d0000000000001976a914af7ca948770bedbbf4c6ae9f0cd7f4f57a03bcef88ac0a300000000000001976a9149bb32a0db659512c6be4da54fd3d6792a50896e288ac479d0000000000001976a914e802b866eecaa95e49e95da8c55ae6a56fecd47e88ac172c0000000000001976a914a975a305b144f67fe6ac3c9cafd80192fc1ed10488ace0220000000000001976a91439e1092a08257cec01a538826e39b500dcccaf3e88ac26410000000000001976a9144b000c344c5b3179cb1bdc74beaf7d07884451d388acf3cf0000000000001976a914acb4ab0b688d9a477f9355c0db5d8a711f5e9dee88ac228c780b000000001976a9140310146cd1252fabe79848e23fc89eb5dbd2519688ac00000000

readcash commented 4 years ago

Ok, talking to @blockparty-sh it seems that in this particular situation a restart was a root cause:

R C: one plausible explanation, however unlikely... is that we did 4 transactions, which went into mempool

node restarts

we try to get utxos, but it is still syncing mempool from outside

it gives us an old utxo

Jt: well this wasnt really restarting at all before (other than rarely/manually for upgrades) and same issue was occurring

R C: we spend it. bchd remembers and allows it, but then it doesn't allow external tx from bch network

Jt: i can slow them down too

R C: is there a way to find out was there a restart at the time?

Jt:

03:05:11 CEST; 41min ago last restart

R C: double spending transaction was sent at 2020-08-19T01:05:29.900951529Z yes ~40 minutes ago

Ok, i'll add it to the issue... I don't think it solves the original issue though

But it seems that we need to stop restarts

I think this is actually a problem of its own (getting UTXOs while node is syncing :( ), but it seems that it is unrelated to the original problem.

zquestz commented 4 years ago

Thanks for all the info, we will definitely be looking into this!

zquestz commented 4 years ago

Next time this happens, dump out the output from https://explore.cash, this uses the same node and can display the tx even if other nodes on the network don't see it!

zquestz commented 4 years ago

The fee for tx was calculated by BITBOX this.BITBOX.BitcoinCash.getByteCount then multiplied by 1.01, then Math.floor.

Here's a similar transaction: https://blockchair.com/bitcoin-cash/transaction/ca7b6388346eed0e29b4e9c3907c813bb09b911c387426cbc1919b7da1d445c6

Size: 3,625 Fee: 0.00003662 BCH. So the failed transactions probably had a similar fee. So, it doesn't seem to be a fee issue.

It doesn't only happen with big transactions - sometimes transactions with just a few inputs-outputs fail. (By "fail" I mean that they are accepted by BCHD, appear in mempool, can be spent, not broadcasted to the network or maybe the network doesn't accept the broadcast, disappear upon node restart)

Is there a way to enable verbose logging in BCHD maybe?

Yes, there is a way to enable much more verbose logging. I am going to enable it tonight on my server. Just start bchd with -d debug

My guess is that upstream peers don't like the transaction for some reason. Have you tried broadcasting any of these transactions through bitcoin cash node or bitcoin abc? Want to help you figure this out! Also how quickly are you sending these transactions? Are they one right after another? Is there a delay between sends?

readcash commented 4 years ago

The problem is that these "bubbles" appear randomly. It's very hard to predict when and why. My hunch is that they might be due to low fees, especially if users ignore the maximum we recommend and set the amount manually (thereby creating a low fee transaction). But I thought #382 was supposed to solve that, but we've seen these bubbles after that too.

The problem with catching "sources" (hex representation) of user transactions is of course with the fact that they broadcast it themselves (via js) directly to the node, so we don't really have any way (currently) to catch and log that. I'll see, maybe I can add something.

zquestz commented 4 years ago

The problem is that these "bubbles" appear randomly. It's very hard to predict when and why. My hunch is that they might be due to low fees, especially if users ignore the maximum we recommend and set the amount manually (thereby creating a low fee transaction). But I thought #382 was supposed to solve that, but we've seen these bubbles after that too.

The problem with catching "sources" (hex representation) of user transactions is of course with the fact that they broadcast it themselves (via js) directly to the node, so we don't really have any way (currently) to catch and log that. I'll see, maybe I can add something.

It would be great if we could get more sample transactions that are stuck. That would ease our debugging immensely!

zquestz commented 4 years ago

I also have CRAZY logging enabled on my node now, so debugging these should be easier and I am logging every INV interaction between my node and its peers!

readcash commented 4 years ago

Thank you, Josh!

We'll have to wait for another report from users. I got some reports today, I replied to get some additional info, but still waiting for responses. I'll inform you when I have anything new. Thanks again!

readcash commented 4 years ago

@zquestz:

f9d1bdd8784cf21cca44d0e70b28c7672843474356f7c5717eac9cced4e600b5
23daaa0b36d886fb50acbc2495b7159c108ef7b9f299880642d1fead387c4957
d1483ed9155d0269b910b2b67664a3373674f1b816bdab5316f77458cecdd308
677e364e4fffd748ea285a056888f65e640aa86f49beafd648540fc3ad1c1758
d01e85d953e7a33154342e70d064a0230951a9aeb5dabba69bb570326192a727
f9d1bdd8784cf21cca44d0e70b28c7672843474356f7c5717eac9cced4e600b5

Happened in last 24 hours. explore.cash can't find either.

Thanks!

I figured out that we have paid upvote TXIDs, so I can check them against rest.bitcoin.com, so I can easily find missing ones, at least here.

readcash commented 4 years ago

Some fresh transaction that happened on BCHD, but haven't been propagated:

^ "[NOT FOUND] 09af27f6960e083c103d0f46ca719f90ce8c26eb313fc95d02ff78945b5aafbd"
^ "[NOT FOUND] 7366ef657c94ab149e6364901f246770be6e237be9ab8fa73224cb7dd8fba78a"
^ "[NOT FOUND] a907b0a9ad20918a5960b156396bdb25957076f805f8891a19873804bc03f156"

zquestz commented 4 years ago

I don't see any of these transactions in my logs even in debug mode.... very strange. Any chance we can start getting the full tx payload so we can analyze those? Also @cpacia if you have some time I would love help debugging this for our biggest user =)

readcash commented 4 years ago

I was hoping we could do without the full TX, since these actually are send browser-to-GRPC, so I have to create a completely new logging channel for this, but since we don't have any choice...

zquestz commented 4 years ago

I was hoping we could do without the full TX, since these actually are send browser-to-GRPC, so I have to create a completely new logging channel for this, but since we don't have any choice...

Me, too. I turned up the logging level on my node to debug and I assumed this would be trivial to track. Unfortunately I don't see ANY logs for these txs...

readcash commented 4 years ago

Still trying to monitor, but haven't got any "bubbly" transactions in the last 24h

readcash commented 4 years ago

GRPC (bchd.fountainhead.cash) replied with TXID, accepting it, but TX never appeared anywhere after this.

Failed to propagate: 62d9a7d1dc7ed88c9ec4d1ba0c5ab91b646453104eea3de6e4f5465ac98fdd78

Source: 02000000171520d878d2cb95ce73d3111daa95c33837f31e08fad561338c0e41f2e2f049cc0f0000006a47304402206b98ec0365eb809aa91f11e34d8dff221e3fb157b4043c9ece9108f7fd889134022009e5e9d3faf4109f58be6f5fd3655c9e4cb9b144ba76fda9d718a454e0d3cc584121033a6b4a6f2d845f889d1be4868e81b53a73f1f94c58b2c31eb5e590a742322f06ffffffff99f5533908868640a7c01ab68f40c17592788b74ccff02b0f2ecdc2befddad6a060000006a4730440220172fd57a065ca8e376043bc8bf699ae1ba199e5269c24c8106ae21bf8b3d58da02202df6ca95b17b3917ecde1a1ba87cbffc39567b8fa85bc4f86379cd1151373d114121033a6b4a6f2d845f889d1be4868e81b53a73f1f94c58b2c31eb5e590a742322f06ffffffff6350d2741805d64a1196f8316460fb43a2bd59398365401b1c2cba181cd124510200000069463043022017cc0f711a01ae2d6f1d35b2489006f512346c692c5ef2a26efa957378f941f9021f15b9f33ca65c62abd10a786d3d31d7e1666d006cf8d047d5ab492960a2f09c4121033a6b4a6f2d845f889d1be4868e81b53a73f1f94c58b2c31eb5e590a742322f06ffffffff04d0b40514c3e29982d7a2c96f7d80fa69e30fd926db47653a7824dad27744d83e0000006b4830450221009309b8e7ecfb46272eabb3ece901bcf8b6d158776c3aacd4b74fc438d88024bf022056ea04f979ecfbc5416d27109e46ceb97a1086fd40a7db75a98ac66d1b5ea5904121033a6b4a6f2d845f889d1be4868e81b53a73f1f94c58b2c31eb5e590a742322f06ffffffff5ba1fdc0cf6a09500368dfbf31714ce069b4dcb705a2200f03e33051863a2d291c0000006a47304402204745afcaf5bfe6b7a9c6d816d2d9e25a402fcbaf64b65ce999e3f5df7b778646022015fd24a928fe89e3ca4fa94dd006eb32c137969bc68c71e15f22b0a2b73815954121033a6b4a6f2d845f889d1be4868e81b53a73f1f94c58b2c31eb5e590a742322f06ffffffff008f5f9aa0f0ad80cbd12181764bf2086075502b12acb5be9f2bbec18becc7ac400000006b483045022100b831b92e6d7f915d6758ad8bb200b7dec0765c075143ff289180c9fe1038b31b022024242f268d8da6d57fa999fcf7e12ac1272fbfe7a246324e188cfbc702b9aa2a4121033a6b4a6f2d845f889d1be4868e81b53a73f1f94c58b2c31eb5e590a742322f06ffffffffad9a6eb3b7f95912a1c2715c3dadde682e4ef5ca313f7d855bc924c48a59f155460000006a47304402207ab484b2016137f73291a44d5f5f8691b6e7715dd81b8216011183a16009cedc022065aa71d592998aeb92347762b13d53ade34fddf0dd4256a4e546d5fce96b878d4121033a6b4a6f2d845f889d1be4868e81b53a73f1f94c58b2c31eb5e590a742322f06ffffffffc4c76abbc91ada130636765d8bf71fcb5fb726e0f11a780c61f9826121e2a46a320000006b483045022100cbcbe4e5c3417a314017973f496a338bf52c786132136a9f0c83d4eb89910798022023bf94888e41241390a56ca8340df2a86e5f28f29cf86834c308cda5b5f790904121033a6b4a6f2d845f889d1be4868e81b53a73f1f94c58b2c31eb5e590a742322f06ffffffffd62cdb10083346492924d48c8d968ce6df0da319aa6b9810d79fb446f7152a0f380000006b483045022100904f5e5fca9ea1ff55bba69d4320f6d39338c883dc89c8b1cb6620217646d33e02203b65e5addcc044401a2c797131dc421d75c20d0c908ad92fe25072f67ef42ed34121033a6b4a6f2d845f889d1be4868e81b53a73f1f94c58b2c31eb5e590a742322f06ffffffffd8e76917e3400af641707fc3bc76af1d4e0aff4eed2fda7ca283e0fe43fdcaf7040000006b4830450221009f7cc3c2af897c319bf12656b349cde53361f8706b623d40016274f19f6b179502206bbf04c254ebce92c01ee97bd1f9aacc7cedfe0920146e0b0769530b405002c84121033a6b4a6f2d845f889d1be4868e81b53a73f1f94c58b2c31eb5e590a742322f06ffffffff34633a345bae928b9c5ae73544f4c73289e74f19e714a7d3e56c65bf089e087d000000006a473044022061928ccb95c426b49105532f08c5908ad89564ca00a3f2fd21ba432d14add69e02205ce1edb9c2e51b39c103bfb8b46df011c6d8c4d19edbfd3181400e841bdc76a44121033a6b4a6f2d845f889d1be4868e81b53a73f1f94c58b2c31eb5e590a742322f06ffffffff2aa2b22dd21bbf5685b550abc5ffa2182900173aff4ce40c9db62f89b993916d010000006b4830450221008a6e38225681d2ada5dcfcc6d415346eccd290838b8fd4ee9c57249a4f9cdce10220440d969b30cd1b198002a8a9f3a441784877fabe0c92cc0dab6e113a5875ffe74121033a6b4a6f2d845f889d1be4868e81b53a73f1f94c58b2c31eb5e590a742322f06ffffffff5d1350baedb3fafe5bbd940514779afcf94dacb2e303f3b2f3f6089e6da0b0fa500000006b483045022100f1f7ddd00c3493b1d8dd5b3106c93cb0c7b010e42f774faf6ca4404b667dfbe9022077e225068790408cf378f9e1c8324924da24f0bede9a4302b698d9d6689cc5654121033a6b4a6f2d845f889d1be4868e81b53a73f1f94c58b2c31eb5e590a742322f06ffffffff913724b0b8fe235f85535cfe5384174ab7a9cf81d14b440a3465947fb929e9f22e0000006b4830450221009c7f37fdd31f2c042e6b206a045f5a992ea6135385cad4c60587a1e53a33c77e022014b7b56c4f9c946aa6b4c758128d7e6c071c25f1454a5d9d21ebe0c7091c1d894121033a6b4a6f2d845f889d1be4868e81b53a73f1f94c58b2c31eb5e590a742322f06ffffffff2eb1f1aa48af7a2a76fb362c63c03dc3f38ce28546f48c3f5f86aaa581e9ad940c0000006b483045022100fbc7ff15350ebc68ac5b4ee6a30ebe12b731050911f9bb25d0367d83d649de4302205118b955bf06fe226b56dad26416e33f244e14bb1f5cde362816b2efa24f20574121033a6b4a6f2d845f889d1be4868e81b53a73f1f94c58b2c31eb5e590a742322f06ffffffff2ea79ba62d39c70b51df75fa199432c51e44ac8fb6854ffc72e5cb9ace2416fb290000006b483045022100d9222b7ebd32de4f963015aa6cb93f181351fbe818a7129d4a6c18e03daeef360220220230aff8f79c6cb89c5f1d31b4edd0a973841f57dec9edfe626af1f06266e04121033a6b4a6f2d845f889d1be4868e81b53a73f1f94c58b2c31eb5e590a742322f06ffffffff1e57d561cabf1ecb658e5fdcfd8bbad08ea18fc48a4378b6d8b2c8379f2f10c7100000006a47304402205c4133415f2c071a38f330dcbfd5a263bbb558846c367f8ee1c370b035562fe50220681fa671e22400cf5e3599766324c7cd78cb3cb13dfef24d3daf0ced2367053d4121033a6b4a6f2d845f889d1be4868e81b53a73f1f94c58b2c31eb5e590a742322f06ffffffff86afcaf16f630df1a500c6c2d3b548bb9977ba242f70a00f8ba5023b19ff3bf5470000006b483045022100bc10e9afbd498adf6e7ca02220711c78e48bbb968a4ac253211bf524a1f2b40a022024950bd895fb374884831dcac4b4960439ccaa292feaa5cb8d7e56d761cab3864121033a6b4a6f2d845f889d1be4868e81b53a73f1f94c58b2c31eb5e590a742322f06ffffffff37c27ec7485480067dfd7c69e20853cd8020f89b621d81afa44858eacce66f050c0000006b483045022100ead052e24b625f975d31546583ca8f2f0ba5e61370993a8065edc466757eb86c02205cdad6da2e0a6d3ed1ec0e26c23905db476a09f7320455e8f4a260bee54ee9744121033a6b4a6f2d845f889d1be4868e81b53a73f1f94c58b2c31eb5e590a742322f06fffffffffdc21984e3591acf70a878da146ea780a568517da716daa1578ae1c3c72b8a0b040000006a47304402202d6ab5c61b9959074793136ece934a3fa1bce798b3aa83441dc77606172822e402206dac7a9c281a2e1dbcccff4bfbea76acaf612b8326138839f3b047d89c76eea64121033a6b4a6f2d845f889d1be4868e81b53a73f1f94c58b2c31eb5e590a742322f06ffffffff608698c96aadc56b33af605a9974c8ecee74e5a14dd896a8676a6a7f69cd942a110000006b483045022100b0db37ff4c8e109c8a4d68417fe828ecb3043535cef6ea8805c7c66ba814def302207e10c3f37b099901c4b86fc5816ae85589312cd79a4793cbf9fac098c1e023dc4121033a6b4a6f2d845f889d1be4868e81b53a73f1f94c58b2c31eb5e590a742322f06ffffffffd8f2d516a64bab72597f9ca611c8c87a0bda7995677aee4e6caf7c57ffe350ab020000006a473044022021f82dc4e38d7af1c9c5f5bd9d37c5a924f033f117953b31cd7c5f3155974d1802205e5546d64934f7c7a1277589918dc7380a8f21a8064895897cb618650043db124121033a6b4a6f2d845f889d1be4868e81b53a73f1f94c58b2c31eb5e590a742322f06ffffffffa81346dd8740a65854e7d17846d8fa6bfe2823a2fcfb97267a5a1aa4e66667382f0000006a4730440220032ec12a4c6d63785dcdcf335bd1da2050e7127a64e2cf353677e4891d6f45e502203e527ad02998a16bfe1dd17fa775078c2cf9bf177455a5a2d92828293df0549f4121033a6b4a6f2d845f889d1be4868e81b53a73f1f94c58b2c31eb5e590a742322f06ffffffff013f7e07000000000017a9144c6de95889002bf73ef8554e52f325625cdc128f8700000000

UTXOS: [
   {
       "txid": "cc49f0e2f2410e8c3361d5fa081ef33738c395aa1d11d373ce95cbd278d82015",
       "vout": 15,
       "height": 645669,
       "amount": 7.03e-6,
       "satoshis": 703,
       "confirmations": 4671,
       "from": "grpc"
   },
   {
       "txid": "6aadddef2bdcecf2b002ffcc748b789275c1408fb61ac0a7408686083953f599",
       "vout": 6,
       "height": 645523,
       "amount": 6.82e-6,
       "satoshis": 682,
       "confirmations": 4817,
       "from": "grpc"
   },
   {
       "txid": "5124d11c18ba2c1c1b4065833959bda243fb606431f896114ad6051874d25063",
       "vout": 2,
       "height": 645102,
       "amount": 3.152e-5,
       "satoshis": 3152,
       "confirmations": 5238,
       "from": "grpc"
   },
   {
       "txid": "d84477d2da24783a6547db26d90fe369fa807d6fc9a2d78299e2c31405b4d004",
       "vout": 62,
       "height": 644951,
       "amount": 1.518e-5,
       "satoshis": 1518,
       "confirmations": 5389,
       "from": "grpc"
   },
   {
       "txid": "292d3a865130e3030f20a205b7dcb469e04c7131bfdf680350096acfc0fda15b",
       "vout": 28,
       "height": 644807,
       "amount": 4.273e-5,
       "satoshis": 4273,
       "confirmations": 5533,
       "from": "grpc"
   },
   {
       "txid": "acc7ec8bc1be2b9fbeb5ac122b50756008f24b768121d1cb80adf0a09a5f8f00",
       "vout": 64,
       "height": 644661,
       "amount": 3.705e-5,
       "satoshis": 3705,
       "confirmations": 5679,
       "from": "grpc"
   },
   {
       "txid": "55f1598ac424c95b857d3f31caf54e2e68dead3d5c71c2a11259f9b7b36e9aad",
       "vout": 70,
       "height": 644514,
       "amount": 0.00022623,
       "satoshis": 22623,
       "confirmations": 5826,
       "from": "grpc"
   },
   {
       "txid": "6aa4e2216182f9610c781af1e026b75fcb1ff78b5d76360613da1ac9bb6ac7c4",
       "vout": 50,
       "height": 644367,
       "amount": 1.509e-5,
       "satoshis": 1509,
       "confirmations": 5973,
       "from": "grpc"
   },
   {
       "txid": "0f2a15f746b49fd710986baa19a30ddfe68c968d8cd424294946330810db2cd6",
       "vout": 56,
       "height": 644218,
       "amount": 7.0e-6,
       "satoshis": 700,
       "confirmations": 6122,
       "from": "grpc"
   },
   {
       "txid": "f7cafd43fee083a27cda2fed4eff0a4e1daf76bcc37f7041f60a40e31769e7d8",
       "vout": 4,
       "height": 644073,
       "amount": 2.635e-5,
       "satoshis": 2635,
       "confirmations": 6267,
       "from": "grpc"
   },
   {
       "txid": "7d089e08bf656ce5d3a714e7194fe78932c7f44435e75a9c8b92ae5b343a6334",
       "vout": 0,
       "height": 642392,
       "amount": 4.49e-5,
       "satoshis": 4490,
       "confirmations": 7948,
       "from": "grpc"
   },
   {
       "txid": "6d9193b9892fb69d0ce44cff3a17002918a2ffc5ab50b58556bf1bd22db2a22a",
       "vout": 1,
       "height": 642388,
       "amount": 8.091e-5,
       "satoshis": 8091,
       "confirmations": 7952,
       "from": "grpc"
   },
   {
       "txid": "fab0a06d9e08f6f3b2f303e3b2ac4df9fc9a77140594bd5bfefab3edba50135d",
       "vout": 80,
       "height": 641338,
       "amount": 0.00087595,
       "satoshis": 87595,
       "confirmations": 9002,
       "from": "grpc"
   },
   {
       "txid": "f2e929b97f9465340a444bd181cfa9b74a178453fe5c53855f23feb8b0243791",
       "vout": 46,
       "height": 641279,
       "amount": 0.00052366,
       "satoshis": 52366,
       "confirmations": 9061,
       "from": "grpc"
   },
   {
       "txid": "94ade981a5aa865f3f8cf44685e28cf3c33dc0632c36fb762a7aaf48aaf1b12e",
       "vout": 12,
       "height": 641134,
       "amount": 0.00056036,
       "satoshis": 56036,
       "confirmations": 9206,
       "from": "grpc"
   },
   {
       "txid": "fb1624ce9acbe572fc4f85b68fac441ec5329419fa75df510bc7392da69ba72e",
       "vout": 41,
       "height": 640988,
       "amount": 0.00088621,
       "satoshis": 88621,
       "confirmations": 9352,
       "from": "grpc"
   },
   {
       "txid": "c7102f9f37c8b2d8b678438ac48fa18ed0ba8bfddc5f8e65cb1ebfca61d5571e",
       "vout": 16,
       "height": 640835,
       "amount": 0.00030082,
       "satoshis": 30082,
       "confirmations": 9505,
       "from": "grpc"
   },
   {
       "txid": "f53bff193b02a58b0fa0702f24ba7799bb48b5d3c2c600a5f10d636ff1caaf86",
       "vout": 71,
       "height": 640509,
       "amount": 0.00013397,
       "satoshis": 13397,
       "confirmations": 9831,
       "from": "grpc"
   },
   {
       "txid": "056fe6ccea5848a4af811d629bf82080cd5308e2697cfd7d06805448c77ec237",
       "vout": 12,
       "height": 640391,
       "amount": 8.68e-6,
       "satoshis": 868,
       "confirmations": 9949,
       "from": "grpc"
   },
   {
       "txid": "0b8a2bc7c3e18a57a1da16a77d5168a580a76e14da78a870cf1a59e38419c2fd",
       "vout": 4,
       "height": 640126,
       "amount": 7.89e-6,
       "satoshis": 789,
       "confirmations": 10214,
3:34
"from": "grpc"
   },
   {
       "txid": "2a94cd697f6a6a67a896d84da1e574eeecc874995a60af336bc5ad6ac9988660",
       "vout": 17,
       "height": 639825,
       "amount": 9.018e-5,
       "satoshis": 9018,
       "confirmations": 10515,
       "from": "grpc"
   },
   {
       "txid": "ab50e3ff577caf6c4eee7a679579da0b7ac8c811a69c7f5972ab4ba616d5f2d8",
       "vout": 2,
       "height": 639707,
       "amount": 0.00097568,
       "satoshis": 97568,
       "confirmations": 10633,
       "from": "grpc"
   },
   {
       "txid": "386766e6a41a5a7a2697fbfca22328fe6bfad84678d1e75458a64087dd4613a8",
       "vout": 47,
       "height": 639679,
       "amount": 1.699e-5,
       "satoshis": 1699,
       "confirmations": 10661,
       "from": "grpc"
   }
]

http://136.243.250.139:8111/ Not found (strange, this is the node that replied with TXID) https://explore.cash/ Not found https://explorer.bitcoin.com/bch/search/62d9a7d1dc7ed88c9ec4d1ba0c5ab91b646453104eea3de6e4f5465ac98fdd78 Not found

zquestz commented 4 years ago

Ok, thanks for the detailed report. It is VERY strange that the node that replied with the TXID doesn't see it in its own mempool! That might give us a hint as to how this is broken. I want to see if there are any code paths that will return the TXID even if we decided not to accept it!

zquestz commented 4 years ago

@cpacia any ideas how that can happen? Are there cases we would return the txid but not add it to the mempool?

readcash commented 4 years ago

One more idea is that it was removed from the mempool later for some reason. But the only reason I can think if is double-spend, which is hard for me to imagine how it happened, since we get UTXOs before each send.

readcash commented 4 years ago

A few more (happened ~10 min ago):

Failed to propagate: 0d55aeb6ac513d6a401393fc36696c3d8785143da3bf0871785ee76c9f4f9acf
Source: 02000000024674d744d8c4ec74ca494f542e931167b1a85996b4d0b965603042e36036a1e1340000006b4830450221008479db169709a3eff363600edba5f0a55f1dd6e034499baf7a4afd831c6a754902205cb8320184d5a74080bea91f90c3a090bd33beb86809b97fd40bc32577e3cc8d412103fdfa2a944531ff7e3e08fd316ea046b9672c5071e4c6dadcb2e9e25523e67109ffffffff8b7749bb1f1bc975ade22e7af00e2cb679b2d0385fe9ff05040cd6bed6695f8e010000006a47304402204fd3ee434399987837643e66806dd32dec055be750d33ea32f80ffdcb45030020220172b50b4a63fb3c3920f89b1a8d62a7d00c4558cc717a33e297d975b1b316498412103fdfa2a944531ff7e3e08fd316ea046b9672c5071e4c6dadcb2e9e25523e67109ffffffff0275120100000000001976a9147b1475493eceb1e88b57ec71d730f161a20db9a988ac140b0000000000001976a91481f986fbe68c2e509b328bb7eb6d9e139daccf0088ac00000000
UTXOS: [
   {
       "txid": "e1a13660e342306065b9d0b49659a8b16711932e544f49ca74ecc4d844d77446",
       "vout": 52,
       "height": 650398,
       "amount": 0.00070771,
       "satoshis": 70771,
       "confirmations": 14,
       "from": "grpc"
   },
   {
       "txid": "8e5f69d6bed60c0405ffe95f38d0b279b62c0ef07a2ee2ad75c91b1fbb49778b",
       "vout": 1,
       "height": 650324,
       "amount": 2.703e-5,
       "satoshis": 2703,
       "confirmations": 88,
       "from": "grpc"
   }
]

Failed to propagate: 9fffde9ac6d863335cef02caff788ed7ef6c15ee8469a530cbe7ff80a7309712
Source: 020000000651d7cd0e828b7355149b27046add94f4c5c1da432739bc9e534a2d23e3843f4d230000006b483045022100af5a82fb987b948e3dff02c2f9db29076afd9b7369d0b93f5f0bb24c88ed7e4702203061cb1a9997ae31108f0a49b6e52dd104081cf64db7509d15be6f05d5e5ebb6412103707378bae3ccd23bfbec90e0ba62676663bfa5f6267b0be0c102a7f7016128b5fffffffff1ca018a60593c226fa0eb7dd0037d4471bedd0a7dc8bd99d97e3e28ff1be926200000006a47304402202647698786e0b0691cc76fbec623e5246da9b09832bd20a106d276e9f850eea402205c2ba87828df29221a4a77b420d0506f5bad6e30938f044da56b639681aa7675412103707378bae3ccd23bfbec90e0ba62676663bfa5f6267b0be0c102a7f7016128b5ffffffff485b9f1dcdbcf0fdefd016d335111c8a3351c490c07a5adabe08ebb07a9424e14f0000006a47304402204d803062cda7ac74bdec703ef5869c9ffa5f7416567a9f98b899b32c9bceeb07022049396a7f0a67c0224540c8ff6dc19d008eb5fbb4be90d0b7bf47be6612bb41fc412103707378bae3ccd23bfbec90e0ba62676663bfa5f6267b0be0c102a7f7016128b5ffffffff3e7065ef232c3b5028cd546e584f55a5785c94dec6531bdb2b846f0c3802273d320000006b483045022100bf170647ab6f32a4319cf84dff1d69f0afbc0f6481db97ed940b71c4de667536022059db5575375ff4b847644aa5b1c4eac6a28d7471d4691a61abe3bf906217d2bb412103707378bae3ccd23bfbec90e0ba62676663bfa5f6267b0be0c102a7f7016128b5ffffffff987d77aab9afe4b550b0d7d02cb48e0d0408d3ff74cffa3fba795eb0a8f1207e300000006a473044022033f3b2f63c4f00e0efa8566b12da2581701f30ce7a5c943e72d4f028617c09e90220764bf3846c8f68bcb7c6c96a80ea95424f2aa865f80d69d431dbe5999aa4b876412103707378bae3ccd23bfbec90e0ba62676663bfa5f6267b0be0c102a7f7016128b5ffffffff34d96c59e0b21c457804aa3ce1ce3072c6126a18c4b1e114156fc1a87e03b364010000006b48304502210094f4cfb021a92551defcbed463aa8034041d30b677bbde17a198970931ab8cf102200b934dce40a016fd28da020b7c9819efc52b03ddc6416df8250fdc6cd1f8a686412103707378bae3ccd23bfbec90e0ba62676663bfa5f6267b0be0c102a7f7016128b5ffffffff0157660700000000001976a914efbffa3657caf90754bb61462b9afeee0570d1d688ac00000000
UTXOS: [
   {
       "txid": "4d3f84e3232d4a539ebc392743dac1c5f494dd6a04279b1455738b820ecdd751",
       "vout": 35,
       "height": 650398,
       "amount": 0.00330033,
       "satoshis": 330033,
       "confirmations": 14,
       "from": "grpc"
   },
   {
       "txid": "26e91bff283e7ed999bdc87d0addbe71447d03d07deba06f223c59608a01caf1",
       "vout": 32,
       "height": 650102,
       "amount": 0.00071103,
       "satoshis": 71103,
       "confirmations": 310,
       "from": "grpc"
   },
   {
       "txid": "e124947ab0eb08beda5a7ac090c451338a1c1135d316d0effdf0bccd1d9f5b48",
       "vout": 79,
       "height": 649963,
       "amount": 0.00027718,
       "satoshis": 27718,
       "confirmations": 449,
       "from": "grpc"
   },
   {
       "txid": "3d2702380c6f842bdb1b53c6de945c78a5554f586e54cd28503b2c23ef65703e",
       "vout": 50,
       "height": 649692,
       "amount": 0.00018413,
       "satoshis": 18413,
       "confirmations": 720,
       "from": "grpc"
   },
   {
       "txid": "7e20f1a8b05e79ba3ffacf74ffd308040d8eb42cd0d7b050b5e4afb9aa777d98",
       "vout": 48,
       "height": 649554,
       "amount": 2.627e-5,
       "satoshis": 2627,
       "confirmations": 858,
       "from": "grpc"
   },
   {
       "txid": "64b3037ea8c16f1514e1b1c4186a12c67230cee13caa0478451cb2e0596cd934",
       "vout": 1,
       "height": 649451,
       "amount": 0.00036302,
       "satoshis": 36302,
       "confirmations": 961,
       "from": "grpc"
   }
]

readcash commented 4 years ago

We don't have any code to generate the TXID, so it's 100% that we got it from GRPC

readcash commented 4 years ago

More:

Failed to propagate: 021ccb13c784c649de631c34f24f139055f58157d09bbf598afc53ee9969d4d6
Source: 0200000003485b9f1dcdbcf0fdefd016d335111c8a3351c490c07a5adabe08ebb07a9424e10a0000006a47304402206551ff0697f3d609f62c9d1d26589f5b9e0ace602bc493c151efb8c2d222e0ff0220260e893e879db41fc352529457db56a4480abd99012933c1e5154d183cb9a156412103277167cb5edfb291a7437f54f702936ff58e5c85e111cc6f294ecb1062473e71ffffffff6132f4f4a9413e2279b32eafc5c9807bad3b6bce5c6a95f2d186e4b1ed73ace55f0000006b483045022100e3d703a2087c3f185fa537581b9e826afbf259a8e49a6de9bdcf31a2a3e0dee50220565806159cd3d8ad7498d4fb13bb181fc71715a4fa6d3ec6c0cf646735d612f3412103277167cb5edfb291a7437f54f702936ff58e5c85e111cc6f294ecb1062473e71ffffffffe0df243ba36f365dd4e9e7418fb10f08c961313932ed4f174bbf14e209a3b250010000006b483045022100fd08f955908889d591e002b464d4eb210cb6dda7e21b168b96ca4cadcbd5dd8302205bb73b9c32a025f36f555f08b10b2f4509b18ef03d6346189eddceca281687d6412103277167cb5edfb291a7437f54f702936ff58e5c85e111cc6f294ecb1062473e71ffffffff02219f0000000000001976a91489c6d82ea0e382a5a60fd6a018d5904cdb7a2e6e88ac20100000000000001976a914617497341dfe12a37e571d07b73d19fd6d55e3e088ac00000000
UTXOS: [
   {
       "txid": "e124947ab0eb08beda5a7ac090c451338a1c1135d316d0effdf0bccd1d9f5b48",
       "vout": 10,
       "height": 649963,
       "amount": 0.00018649,
       "satoshis": 18649,
       "confirmations": 449,
       "from": "grpc"
   },
   {
       "txid": "e5ac73edb1e486d1f2956a5cce6b3bad7b80c9c5af2eb379223e41a9f4f43261",
       "vout": 95,
       "height": 649834,
       "amount": 0.00023815,
       "satoshis": 23815,
       "confirmations": 578,
       "from": "grpc"
   },
   {
       "txid": "50b2a309e214bf4b174fed32393161c9080fb18f41e7e9d45d366fa33b24dfe0",
       "vout": 1,
       "height": 649795,
       "amount": 2.928e-5,
       "satoshis": 2928,
       "confirmations": 617,
       "from": "grpc"
   }
]

readcash commented 4 years ago

Got dozens more today. Latest example:


Failed to propagate: c721030bd44126e57aacce1979e8ca93fef3d4a6c26f2bc5f8f77e989e3d94f0
Source: 02000000021027fc59a9dec504cf413e054b36500c2d6c74841afc870d330e55e3e1fa49c8300000006a4730440220597d44749e3174460d12b08126e70e1ab5fd779c84e410fc74f57c134876184202205350f86a4ce3920b6f235740871fa9dc5abb5a16b5947cde77ecd04bfed8bc8b4121020d41565a9b1e317665e6e35e93d577a4737db9849e56900e7876da141302582cffffffff3a5a15a1efa430fb7ff5d03246dbeb7ee46e3a8fdb4a9a820365453fe33262be010000006b483045022100eb84d8676fff0e4ae2c38b89f08c447ce007973c948f7c92bff8dae15e57ad8b02207cb8b817a354baa5e07c9bb347ad45000c2d917d0fdee0e2c088c9d1fb8656334121020d41565a9b1e317665e6e35e93d577a4737db9849e56900e7876da141302582cffffffff026f0e0000000000001976a914264dd9318affe28a856d901c68ad8e053def38f188acd9070000000000001976a91450f26c1f80ba124c0614f6dbd24da58a0433c30c88ac00000000
UTXOS: [
   {
       "txid": "c849fae1e3550e330d87fc1a84746c2d0c50364b053e41cf04c5dea959fc2710",
       "vout": 48,
       "height": 650550,
       "amount": 2.731e-5,
       "satoshis": 2731,
       "confirmations": 23,
       "from": "grpc"
   },
   {
       "txid": "be6232e33f456503829a4adb8f3a6ee47eebdb4632d0f57ffb30a4efa1155a3a",
       "vout": 1,
       "height": 650448,
       "amount": 3.35e-5,
       "satoshis": 3350,
       "confirmations": 125,
       "from": "grpc"
   }
]```

readcash commented 4 years ago

Tons more. People constantly complain. We need to figure this out.

I withdrew $0.74 worth of BCH but it not showing on the blockchain? What happen?

I try to cash out $2.80 ,but didn't yet received almost 3 hours.correct my bch wallet

Greetings from my side. Please I tried boosting my article and it showed sending..... But it never showed and I lost the amount I wanted to boost with. Kindly help me out.

The balance displayed at read.cash is still higher than what I show in Electron Cash.

Note: $0 is when getUtxo fails for some reason

Failed to propagate: bd0acfeac9e798c83e8b2760d4faf106301b6790d37f47f184ec43bff5d46157
Source: 020000000144d80b5d8bc2470186f0d6463e88a7fd716c8dbf331b35b4e87c9147a942ee8f010000006b483045022100e4e7d432462a37b756dd899ea1fd997567d178d7ca6c66ef61f2a0b0caaecd77022019b37f37b4e35d63d92496320f2e8f1c33cfdcbbae95d70c5e60f1c107827ab34121035c2466fa7c4ec509ef628a832f0bd15c35603e3fbfa7e5483eb8e8ab975a8f3affffffff02400e0000000000001976a914038c76d20c8909aa4963b6c81ccd2338264df50888ac076c0300000000001976a914ba550eed4835a029be1a2893546bce6f259ab7bb88ac00000000
UTXOS: [
   {
       "txid": "8fee42a947917ce8b4351b33bf8d6c71fda7883e46d6f0860147c28b5d0bd844",
       "vout": 1,
       "height": 2147483647,
       "amount": 0.00228139,
       "satoshis": 228139,
       "confirmations": -2146833019,
       "from": "grpc"
   }
]

4:08
Failed to propagate: f89484508c4a70650e8b99363a2f2987af2d8b0f0e0adb7e759e1d9c918f13af
Source: 02000000015761d4f5bf43ec84f1477fd390671b3006f1fad460278b3ec898e7c9eacf0abd010000006b483045022100b702fa59c0218ec6a21b5ff58610fa5c885282a5f7bfe8a763f024e03d7c684202207d0a6e784de74adb4970facd91b4dfd0d7c2495492025b28b631142a010357024121035c2466fa7c4ec509ef628a832f0bd15c35603e3fbfa7e5483eb8e8ab975a8f3affffffff02400e0000000000001976a91407291cffcb56e91b4b420e4f24359469802b930788ace35c0300000000001976a914ba550eed4835a029be1a2893546bce6f259ab7bb88ac00000000
UTXOS: [
   {
       "txid": "bd0acfeac9e798c83e8b2760d4faf106301b6790d37f47f184ec43bff5d46157",
       "vout": 1,
       "height": 2147483647,
       "amount": 0.00224263,
       "satoshis": 224263,
       "confirmations": -2146833019,
       "from": "grpc"
   }
]

4:08
Failed to propagate: 6725ba5cde7f816ee391e56eeec8cf0ff174710ae46ba9d20e3dee7f32dd549b
Source: 0200000001af138f919c1d9e757edb0a0e0f8b2daf87292f3a36998b0e65704a8c508494f8010000006a473044022067ea615ac810c2613e0d154b0a15d4710cbb7840d249cf537a851f3e6de009ab02201398945ef08acbe118669dd2cab4bd36615addd1da44029a088b68f74169fd9e4121035c2466fa7c4ec509ef628a832f0bd15c35603e3fbfa7e5483eb8e8ab975a8f3affffffff02400e0000000000001976a914cff3477423d6d788f01dd2e79d1f8c85e947959f88acbf4d0300000000001976a914ba550eed4835a029be1a2893546bce6f259ab7bb88ac00000000
UTXOS: [
   {
       "txid": "f89484508c4a70650e8b99363a2f2987af2d8b0f0e0adb7e759e1d9c918f13af",
       "vout": 1,
       "height": 2147483647,
       "amount": 0.00220387,
       "satoshis": 220387,
       "confirmations": -2146833019,
       "from": "grpc"
   }
]

New
4:08
Failed to propagate: 530be7f036d3e1d9e335eb08da77d0007d4f981edd68f9aae1e71dfe3f1973e9
Source: 02000000019b54dd327fee3d0ed2a96be40a7174f10fcfc8ee6ee591e36e817fde5cba2567010000006b483045022100e97e6f7d967de6724303c1056698e83028bea992fabdffbc59b711052d789c5402206636d13e5aa87dbbb4f1ef926eea59dc5eeec30745ed468279d6f561eb34a8934121035c2466fa7c4ec509ef628a832f0bd15c35603e3fbfa7e5483eb8e8ab975a8f3affffffff02400e0000000000001976a91417a9d399ae2654e1b2367f0b70c24748cb8f553b88ac9b3e0300000000001976a914ba550eed4835a029be1a2893546bce6f259ab7bb88ac00000000
UTXOS: [
   {
       "txid": "6725ba5cde7f816ee391e56eeec8cf0ff174710ae46ba9d20e3dee7f32dd549b",
       "vout": 1,
       "height": 2147483647,
       "amount": 0.00216511,
       "satoshis": 216511,
       "confirmations": -2146833019,
       "from": "grpc"
   }
]

readcash commented 4 years ago

Another example:

Failed to propagate: 2a4bfab6d9de41b6f834638447171e0392860cdba4d75a967d3ba30445fcafed
Source: 0200000002f77da22a29d81c237bc7721088bc76f50433eac019a1f2635fab080f35bdcc2c010000006a473044022032ba2eceeea712229b915fe7ec49fcdc87e44dc7b035aac983d0db6843b3650202207402e54f58be8cd9489fdc2406114ed4a44929a04d05effe4b36ee25ea4b90c34121025c8996d1de6ad7f97822160ade563d71ad10dbe8fca89a437c84241fe0cab38fffffffffbad9a5081f79e1e534fae56d9fc6113a7d97d8250eed4efc9717ce2cb65ca4bf4a0000006a473044022072adad14820c6ff331a685dca4714ae47f81b817809b92079068574084e2a5f602201a62cce2abb0b3c96b6c2c56a49bb2c2f366d5cbff777cb1977bad9c3dd76cfc4121025c8996d1de6ad7f97822160ade563d71ad10dbe8fca89a437c84241fe0cab38fffffffff02e02202000000000017a914e5e6607b4c31c6659bfc671b1fb09f05f53c229b87330d0000000000001976a914d71f4c701c581ae009421244609a4f1136db79a588ac00000000
UTXOS: [
   {
       "txid": "2cccbd350f08ab5f63f2a119c0ea3304f576bc881072c77b231cd8292aa27df7",
       "vout": 1,
       "height": 650553,
       "amount": 4.041e-5,
       "satoshis": 4041,
       "confirmations": 144,
       "from": "grpc"
   },
   {
       "txid": "bfa45cb62cce1797fc4eed0e25d8977d3a11c69f6de5fa34e5e1791f08a5d9ba",
       "vout": 74,
       "height": 2147483647,
       "amount": 0.00139715,
       "satoshis": 139715,
       "confirmations": -2146832950,
       "from": "grpc"
   }
]

http://136.243.250.139:8111/mainnet/tx/2a4bfab6d9de41b6f834638447171e0392860cdba4d75a967d3ba30445fcafed (in mempool!) https://explore.cash/ not found https://explorer.bitcoin.com/bch/search/2a4bfab6d9de41b6f834638447171e0392860cdba4d75a967d3ba30445fcafed not found

Went to https://rest.bitcoin.com/#/rawtransactions/sendRawTransactionBulk Entered the tx, got

  "error": "Missing inputs"
}

So TX is in the mempool, but not propagated

Parent bfa45cb62cce1797fc4eed0e25d8977d3a11c69f6de5fa34e5e1791f08a5d9ba:

02000000057b35e093f47c4bbc80d5b682d87a31fc52828a2b53b540a308839f5697ec8e5e650000006b483045022100bd3933d50fe81780d39e8f419a996bbd26476d9e839d91aa24be9c0428d48b47022005ef9197bf620f47fd7c4b8fd124380c505dea84d43c91ee7d0680084408ec2e41210337d4dfb6295828b4e573e69a2e13ec564df92703a8adffa9d0de7c3de45023f4ffffffff9584b963625f3fbe1d982a362401d4763e5a2ef1d3d5afb5b05007b63e8a1aa5650000006a47304402203cb9a905449722ca6a2c0a875c2bf3fca8ad4b81aef8f3f231fed14b380dd3de022021f73b5a0de92c9ecfcbc6805bb8b5cd24869d712f197fee72bfb87fd23fc66e41210337d4dfb6295828b4e573e69a2e13ec564df92703a8adffa9d0de7c3de45023f4fffffffff3248d5c5fc899e366027c6b472a4dc418ffedddfb47991556faa6004bc03a76650000006a47304402202616ba7670bcb95ec057fc2bb19c094a6dc2e57da7fc480160f2f87ca8b993ca022052bc674907b0bf5472865817a1d9732919fce893c89b976ca89846ae5924c47b41210337d4dfb6295828b4e573e69a2e13ec564df92703a8adffa9d0de7c3de45023f4ffffffff8e7cd0d493fc68113458e8e9fb50652071fccb84efc829affccbe0ffc49919d1650000006a47304402203a61876d3ce2be87237eed42f45e8897d76659a7db77e8c7a04802c4e69f10ec022045dbff0b9b90a3dbc006ba69fde11fff6f06b331bc8b15e171ce6891e8373b1941210337d4dfb6295828b4e573e69a2e13ec564df92703a8adffa9d0de7c3de45023f4fffffffffb5f475a09432e7fb2593d99e04e4bef74a2ac43d8986f833daaa8fca99de0b6630000006b483045022100ec94aee074cd18618d18bbfc08112a69f161ec2bd5bf606274f51a6c200e2629022036fccc46b7d6b63863adb0cf24516dc1d578165571a5c8eb7bdf93c37e3bfd0e41210337d4dfb6295828b4e573e69a2e13ec564df92703a8adffa9d0de7c3de45023f4ffffffff653dab0600000000001976a914eee133dc47ab88fc982115753ffcff49e1191b4e88acdea00100000000001976a91468ef746d86bbf198c606bbd1af037b09bde1145a88aceeb70000000000001976a914e523ca5740be346c6336e2d63080e61789484a8288acbf060000000000001976a91476e42ef4ae5083e4ae8b828caf686a16577cef8788ac39e10700000000001976a9147bc6eb8ad849a2eee500f8e9af65961a5885486088ac55100000000000001976a914d6a44d84af00016444188db0fa0e099f47010ef388ac0b110000000000001976a914f40e3c4f55a50163e5a0eb4ab693040e9749a79c88ac96dd0100000000001976a9148ce78ff438abc8b71271018fef01c0efecae4e8388acbff00000000000001976a914049485999a37cb6d337cad2756cb26efccef279e88acf6bc0400000000001976a9147b6348d048f6ebb37334323c359aff6f76c122bb88ac5b310000000000001976a91451d12e7020869296132a1a77a79c2c6aa34a341588ac61880200000000001976a91440a05bb3f58b82763d1faafb057df645c86725ed88ac32030000000000001976a914dcf7321bad89e696f2f46bd6ddf509b3912bd37288ac21330000000000001976a91459a0bc71da98fa62e78f8ba2a6cbb020c5b3613988ac95910000000000001976a91459d01f77bb67cb715749f512b405cf1528cf506288ac0e9e0300000000001976a91437c3c6c1b97ccef1592e33ae2c48848175525fbc88ac91340100000000001976a914d294c0b42f236fe33f84011de7c6f639ac1ae06d88acde020200000000001976a91482e54486de5b535f0dced294bbeddf97dd31fddd88ac363e0500000000001976a91418c717b64b2c2ba52ad79a46745621b0fdccc5a988ac3c690900000000001976a9146b6da3dd2aa97e80ba8bdf24a06eb7d17cd7d80588acdb490000000000001976a91474912737675b31bad6a8d30498df943e7fdc90ef88acd2480200000000001976a91489963c57819667005272bf19b498d08fe0a1fb1888acc7af0100000000001976a914599616fd641f05503af0df57e75b75d464a8d19688ac182d0000000000001976a914b7e7d6a1e3a38e30dd9dd390e6b096d0d954942988ac6d0c0000000000001976a9142d1a8859f231e65ec1802c17c71672a28a2f2b7388aca70a0000000000001976a914a3fc276b8874e5f7d229c2cdf36722d771eac0fc88ac3b090000000000001976a914ccc208d6713344e8de9f3aeef370201c09d4818f88acb9160000000000001976a914d8bd90a2665875d6ce1156d85aeb45cc919d33c388ac26dc0000000000001976a91460dcc597259fcd8e15750bb3a9554a321b8c619288ac9bfe0100000000001976a914ab212e1f9890fed7a13765e4520f3dd87b7e288a88ac9ff90000000000001976a914bc55a2708af2b9477f42194f96396dccded5956288ac196d0600000000001976a9147fe247073722548d6555d16cbfcd2a9d220545df88ac53050000000000001976a914db8c7af057d535193c06e47bffe0e0742a891a2988ac1c2d0100000000001976a914f587eb22f34585e6c8ece976539de85a67e96edb88acaf9d0200000000001976a914f60b4ab2c7d6238728361e684c05ec332c5d542988ac2a080000000000001976a91436a1c586381cec45e3af53d133d9717bdab3f02788acd5280000000000001976a914d14aad4fe36e90eae2a69aad877871e4d56388ff88ac25180000000000001976a914164a42f2e362d027ea8d3ffb20280154446927aa88ac412a0000000000001976a914f688ea5b9dfc7001921fafff602cd448b98f7c3f88ac0a990000000000001976a914dea2802582c854d2d99a45b7a6172777a373972a88acd6a50100000000001976a914c65506407d49ad01f57699c69861f1e52b32126888ac4f0a0100000000001976a914431365597da212dfb18755e2325648632ccab91a88ac305a0000000000001976a9142b6a9c5b8638c7a7f6ec284348d4b96685e5974c88ac230d0000000000001976a914ca65847d8e03b0806c9511c7c91e882adc4d3f8388ac40e30000000000001976a9148e2762c3deead3f1257065504c6014af7f50942c88acf1090000000000001976a9146037c292f19cc276cf302bf53b82f07e6a8004cf88aced550000000000001976a914f43bfa865557dcb553a0fe04151dd38b8253929d88ac7ee10100000000001976a9147193f41da470704666e3467ab30cb79e85bd6b7488ac5d720200000000001976a9147ad71c353176f86578241da1d31eb32f99c1b9ec88ac230d0000000000001976a9149514e42866e9ac3d530b1a28fc4b06095be1776488acbd5d0000000000001976a9142a3126bcaaac9dffe7aa087af66602e74ee7ed7f88acf6e80200000000001976a914084c96d827fa446f5878c179248cceea79c10a2d88acaae80600000000001976a914347e779a3ccbcb95c10f05b358f36bce2bf6588588ac528d0000000000001976a9149cdcaac5208e30c1037679f3bcf8246e2f16e6bd88ac9a0e0300000000001976a9144ef76af6dbd6ff7169650afcc448e68fb3f19f8e88ac292e0000000000001976a91475b666a37514d4086dc2fe6e6056e641559534c088ac6de00100000000001976a9146bd0f8b4dcd1f129c04fa650f83b4157348ebf5b88acda020100000000001976a9144bae590dbda482f0efaca3c38b754e0c7b15319288ac1fe10000000000001976a91439e1092a08257cec01a538826e39b500dcccaf3e88ac0b110000000000001976a914c4023b5a28def7ea7b5ad73f7c4647c78716f03188acc5730100000000001976a91416d8569ea3f8e170a86d380e902bedbe1495138188ac4e1a0200000000001976a9144240788141f01113b278922dc962721dac7c93aa88ac88490200000000001976a9146948bdb65b76155dca2267bdbc1282dfa820bb5c88acdd740300000000001976a914148e77bc98fa38f8d7bb9f2ac333edb28b86cc2888ac70790000000000001976a914fdd8b50d8ab946f41fadaacb85330a964c747cc688ac340e0000000000001976a9149288b0ce49ea933aafd5bf330337cb7fd01b991d88ac1e840200000000001976a9140bdf98e0413e29a6c91adc61173a440565c7088488ac8d650000000000001976a9140056ddfa3420b747ff4b4b123a661c5e575d7b0488ac17b50000000000001976a91414cec388b8d4ed8ef26284c0156e6267e8c6d21488acfc1a0000000000001976a9148eb514c21f32e808a0263e98a3d70e03b24a333688ac8f5a0100000000001976a9140ca81ee972a39d43df187495c542282e312650b088acb1500600000000001976a9141d607c63db7e2b9cd15bc2c6f9264ad8d863ed5088ac78220200000000001976a914e3e32eaeb9daf38abd5744ee11a934550819a17688acf5810000000000001976a91481aa9053c2f479e5eb5adb025ba40c2f8d72ea2288acc3210200000000001976a914d71f4c701c581ae009421244609a4f1136db79a588ac185e0000000000001976a914b3f05723a7c12ac20105448ade863ae452c7677d88ac461a0000000000001976a914d9be6ba22acab719bcf909d913b39488312094d688ac72b50000000000001976a9149f587123d18c100eb2339d61417d71d2c2d02f2988acb4610400000000001976a91497542ee4418586e96cc8ccc1f7b21f848a81d2a988acd7ec0000000000001976a914f8da796905fd5012a925bc5a35b405750d2029c088ac530a0200000000001976a914c423af6f257b925f8222ef7c0624ffa808f56db588ac7b740100000000001976a9141f340bdb9109643d0f46e0fe5af7efed3c68308788ac59260000000000001976a9141bddc1502cabc188377f8c251cfb2f22d803d94e88ac010b0000000000001976a914a05fcda641788861a2c1ca70d5a776c5a46e7fd988ace7f20200000000001976a914d55b24f65ae0c770614b001c38ee13dcc490a66088ac40810000000000001976a9142960746db630760d277a6d7772e891e3bfdb44f388ac7d840300000000001976a91480f5fe10c6e02b55b2588f1fe9746f5944aeeb1588ac5c0b0000000000001976a914f83363584aa41ec86de9ebb658f283d952fd47f088ac204d0500000000001976a9144360d5eb4133e47bcc00976009b598d4ee4863c888acc1110000000000001976a914d59c86f946a384a99a4fa1827618568d7a3f319e88ac84640200000000001976a9142514782c04dc789edd276eef9b127707781c870b88ac23f70000000000001976a9148ae11db93a63551ff0fcc3961d944dfd5dbc0ad888acbd2c0000000000001976a914093038c983ce5b9afb8c1df20b79b02113a41c9088ac55d30400000000001976a91470fb7e882deafe659b373c654d79a04a2839760388ac7e3e0000000000001976a9148a741e007acd181b0e8ffd11099b133638ad428888ac53200100000000001976a914fa6323e92b47bf9116ef8719f28d3bcd3434a1e788ac65990000000000001976a9145bb8fdd2e95be3dcdf5c61d1d8bc592217aee02b88acffb80000000000001976a9146b4147100839a476fcd003067ad567cceef76f1988ac77430000000000001976a91426b9915714a55cf57c12170acc3dcf35c9444d3b88ac1e4e0000000000001976a91400dfd9cd63af620459cca66c63630bf219a2c06688ac11669f37010000001976a9140310146cd1252fabe79848e23fc89eb5dbd2519688ac00000000

  "error": "Missing inputs"

http://136.243.250.139:8111/mainnet/tx/bfa45cb62cce1797fc4eed0e25d8977d3a11c69f6de5fa34e5e1791f08a5d9ba

The problem here is that https://explorer.bitcoin.com/bch/tx/5e8eec97569f8308a340b5532b8a8252fc317ad882b6d580bc4b7cf493e0357b :101 is seemingly already spent by 006afe07a5ba6a2a89f5625c1d681fd23d4fcb3e6cdb0af0e2ab6e6181bda5f7

5e8eec97569f8308a340b5532b8a8252fc317ad882b6d580bc4b7cf493e0357b was sent at 2020-08-31T00:17:40.084770650Z

006afe07a5ba6a2a89f5625c1d681fd23d4fcb3e6cdb0af0e2ab6e6181bda5f7 was sent at 2020-08-31T00:18:02.177417501Z (20 seconds later) spends 5e8eec:101 <-- got accepted by the network! 3 confirmations

bfa45cb62cce1797fc4eed0e25d8977d3a11c69f6de5fa34e5e1791f08a5d9ba was at 2020-08-31T00:23:40.815631338Z (5 minutes later) spends 5e8eec:101 (got the same UTXO, spent it successfully, this one is in the mempool now, not evicted)

and bfa45 has a lot of successful spending transactions... :(

I would need to restart the node, since there's now about 50 transactions that spend descendants of this UTXO, which doesn't exist!

Hello,@read.cash ,my name is [cut]. I am a @read.cash regular user. My @read.cash account name is [cut]. There is a problem in my account.And I also face many problems in my [cut] account. Yesterday, 30 August 2020, I lost my 4.84$. This 4.84$ I never got back. Today,31 August 2020,when I sent the dollar money to another account, it showed me the success of the send money. But the dollar was not added to his account of the person to whom I sent the money and the dollar was not returned to my account. Today I was sending 2.01$. Today's send money Transaction: cda2b1ff9b78ce3fc3c4bd7f8e521be4aed09094

In two days I lost a total of 6.85$. Now I am very shocked

readcash commented 4 years ago

You can close the issue if you will, read.cash no longer uses BCHD.

zquestz commented 4 years ago

Not going to close it. Going to still try and figure out why you had these issues but other broadcast heavy apps have not... Just curious what are you using to track utxos now?

readcash commented 4 years ago

BCHN + Fulcrum + Insomnia

gcash / bchd

Mempool out of sync #400