[master] Testnet activation of bip141 failing?

JasonCoombs commented 6 years ago

I am still debugging this, obviously, but after adding debug logging to chain/script.cpp as follows:

bool script::is_pay_to_witness(uint32_t forks) const
{
    const auto this_id = boost::this_thread::get_id();
    // This is used internally as an optimization over using script::pattern.
    // The first operations access must be method-based to guarantee the cache.
    bool bien = is_enabled(forks, rule_fork::bip141_rule);
    if (bien)
    {
        LOG_VERBOSE(LOG_SYSTEM)
        << this_id
        << " script::is_pay_to_witness bien";
    }
    else
    {
        LOG_VERBOSE(LOG_SYSTEM)
        << this_id
        << " script::is_pay_to_witness mal";
    }
    auto bops = operations();
    bool biwi = is_witness_program_pattern(bops);
    if (biwi)
    {
        LOG_VERBOSE(LOG_SYSTEM)
        << this_id
        << " script::is_pay_to_witness biwi";
    }
    else
    {
        LOG_VERBOSE(LOG_SYSTEM)
        << this_id
        << " script::is_pay_to_witness biwi is false";
    }
    return bien && biwi;
}

what I see in the debug log is that is_enabled(forks, rule_fork::bip141_rule) always returns false.

this would explain why @thecodefactory has been able to sync past block 872,730 https://github.com/libbitcoin/libbitcoin-node/issues/381

my own testing confirms that when using the master branch as it exists today, with whatever problem it has right now which causes bip141 to fail to activate for Testnet when requested by config settings, indeed the sync makes progress far beyond block 872,730 and I have almost sync'ed to current on Testnet. without bip141 active it may well sync current, because true block validation is not occurring.

JasonCoombs commented 6 years ago

additional debugging detail: after confirming that the source code in the master branch is in fact NOT doing Segwit validation, because bip141 activation is not happening despite the config file requesting it, I have now traced my development codebase to the point that I can see where the mistake of implementation might be. I don't understand it yet. here's what it looks like in my log:

23:36:22.621731 VERBOSE [system] 0x70000ce39000 script::is_pay_to_witness biwi
23:36:22.621739 VERBOSE [system] 0x70000ce39000 witness::extract_embedded_script() out_stack.size != 2 || !is_push_size
23:36:22.621769 DEBUG [blockchain] Verify failed [872730] : invalid witness
 libconsensus : false
 forks        : 16382
 outpoint     : 1b9e1323923508ecc3d097f11aeaa66acbe6a6d3c4ea6b25d57d34ee9c8d660d:0
 script       : a914e5a679f81579cbf20bb2a4764be9e95e8b3e396787
 value        : 100000000
 inpoint      : 118c83b2c7aa4b500e0c0333dbfef990ca6c324004d68f68acb10ddada918474:0
 transaction  : 01000000020d668d9cee347dd5256beac4d3a6e6cb6aa6ea1af197d0c3ec08359223139e1b00000000171600143f0402fcb8beef28a89be7e94794ec66d469fdb6feffffff0d668d9cee347dd5256beac4d3a6e6cb6aa6ea1af197d0c3ec08359223139e1b010000006b483045022100eaceee00202693d7d54aa84bd8b216f62b1f041b16e08e0352eb1ed50f1ef10102201a90288f667f975cc8d524431d61bd8b15f5d189610d31a9bd12ee37c0674f06012103c4561ce27291b1730e5a429934f45fb4ff9b56e24b6e10e0d176f198fa029671feffffff02506c9800000000001976a9140af575373dad17150d91b6a191a371b59e09d87f88ac001c4e0e0000000017a91403572c975aec5228c0e2982ef8a03b6830e0554f8719510d00

I originally expected the problem to be here, because, obviously this is going to return false when we have a protocol-valid transaction with a mix of inputs, one Segwit and one pre-Segwit, such that there will in fact be a zero-item witness block:

https://github.com/libbitcoin/libbitcoin/blob/master/src/chain/witness.cpp#L444

and, presuming the failure to be happening because of the encoded requirement for a non-zero witness also matched up with the discussion found here that originally alerted me to the possibility that we have implemented bip141 incorrectly with respect to this case, the legacy mixing of inputs.

https://github.com/libbitcoin/libbitcoin-node/issues/381#issuecomment-425723961

but as shown in my debugging output, above, the validation process never reaches line 444.

either out_stack.size() != 2 or !is_push_size(out_stack) and I don't know which it is yet. see:

https://github.com/libbitcoin/libbitcoin/blob/master/src/chain/witness.cpp#L432

somebody owes me an apology. @evoskuil

JasonCoombs commented 6 years ago

I just confirmed that v3 did the is_enabled(forks, rule_fork::bip141_rule); correctly, just as in my v4 development branch. as expected, after its fork activation time v3 detects bip141 correctly. I do not yet know what changed with the v4 code causing bip141 activation to fail to occur.

also, it turns out that the out_stack.size() != 2 is the condition being encountered which results in the invalid_witness error. when stack size is zero for a non-segwit input in a mixed tx any calls to is_push_size() appear to handle that case properly, since the test being done is simply less than or equal to, and zero is always less than the maximum size. see:

https://github.com/libbitcoin/libbitcoin/blob/master/src/chain/witness.cpp#L344

it's unclear why v3 does not produce invalid_witness in this situation, as there does not appear to have been any change between v3 and v4 in extract_embedded_script()

v4: https://github.com/libbitcoin/libbitcoin/blob/master/src/chain/witness.cpp#L416-L470

v3: https://github.com/libbitcoin/libbitcoin/blob/version3/src/chain/witness.cpp#L416-L470

JasonCoombs commented 6 years ago

more detail regarding v3:

the default bn.cfg file includes a checkpoint up to block 600000 here:

https://github.com/libbitcoin/libbitcoin-node/blob/version3/data/bn.cfg#L131

when Testnet configuration is used, block validation doesn't begin until after this checkpoint. the following debug log lines, including my logging additions shown above, show clearly precisely what happens when bn transitions from block 600000 to 600001 at which point validation commences:

12:39:22.859182 VERBOSE [network] Received block from [173.212.206.6:18333] (505 bytes)
12:39:22.859424 DEBUG [node] Connected block [000000000000624f06c69d3a9fe8d25e0a9030569128d63ad1b704bbb3059a16] at height [600000] from [173.212.206.6:18333] (1073742079*, 4).
12:39:22.859447 INFO [blockchain] Block [600000]    2 txs    2 ins    0 wms    0 vms   23 vµs   11 rµs    8 cµs    3 pµs    1 aµs    1 sµs   38 dµs 0.000000
12:39:22.859478 VERBOSE [network] Received block from [173.212.206.6:18333] (505 bytes)
12:39:22.888281 DEBUG [node] Connected block [0000000000022d279bb4239c0a703d12fc6f37f6b9e8e8c196fe2a0692859811] at height [600001] from [173.212.206.6:18333] (1073742079, 4).
12:39:22.888309 INFO [blockchain] Block [600001]    1 txs    1 ins    0 wms   29 vms 28535 vµs   10 rµs   11 cµs   17 pµs 28492 aµs    2 sµs   98 dµs 0.000000
12:39:22.888348 VERBOSE [network] Received block from [173.212.206.6:18333] (262 bytes)
12:39:22.888601 VERBOSE [system] 0x70000c35d000 script::verify()
12:39:23.007743 VERBOSE [system] 0x70000c35d000 script::is_pay_to_witness mal
12:39:23.007777 VERBOSE [system] 0x70000c35d000 script::is_pay_to_witness biwi is false

at the first block above the highest checkpoint height we now do validation, which calls script::is_pay_to_witness() and, because block 600,001 is below the block height at which segwit was activated on Testnet, the result is the correct determination that is_enabled(forks, rule_fork::bip141_rule); returns false.

JasonCoombs commented 6 years ago

further to the v3 comparison, here is what appears to be the activation point of segwit on Testnet, the point at which script::verify() detects the requested soft fork based on the activation time for bip141:

NOTE: in the debug log output shown below between block 834608 (with 8 txs) and block 834663 (with 2 txs) there are 55 empty blocks that appear to contain only the coinbase transaction. I don't know which block marked the activation point for bip141, it depends on the timestamps, see:

For Bitcoin testnet, the BIP9 starttime will be midnight 1 May 2016 UTC (Epoch timestamp 1462060800) and BIP9 timeout will be midnight 1 May 2017 UTC (Epoch timestamp 1493596800).

https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki#Deployment

there is no script verification process involved in validating a block with only a coinbase transaction.

block 834,663 appears to be the first block after segwit activation with 2 or more transactions:

19:30:09.196518 VERBOSE [system] 0x70000e1e9000 script::verify()
19:30:09.196709 VERBOSE [system] 0x70000e1e9000 script::is_pay_to_witness mal
19:30:09.196762 VERBOSE [system] 0x70000e1e9000 script::is_pay_to_witness biwi is false
19:30:09.197037 DEBUG [node] Connected block **[00000000000a1b0b5293090c60dccb3b7af6a8f0009f668b7a5caaaa91063246] at height [834608]** from [47.74.21.139:18333] (1073743871, 4).
19:30:09.197064 INFO [blockchain] **Block [834608]    8 txs**    8 ins    0 wms   57 vms 7106 vµs    7 rµs    7 cµs 6966 pµs    7 aµs  119 sµs   20 dµs 0.000000
19:30:09.197106 VERBOSE [network] Received block from [47.74.21.139:18333] (2213 bytes)
19:30:09.197345 DEBUG [node] Connected block **[00000000003406d957e221d4bd99a421608ad88aa024d26ff494b4268987a25b] at height [834609]** from [47.74.21.139:18333] (1073743871, 4).
19:30:09.197367 INFO [blockchain] **Block [834609]**    1 txs    1 ins    0 wms    0 vms   41 vµs    9 rµs   10 cµs    7 pµs   11 aµs    1 sµs   54 dµs 0.000000
19:30:09.197414 VERBOSE [network] Received block from [47.74.21.139:18333] (212 bytes)
19:30:09.197566 DEBUG [database] Output cache hit rate: 0.162298, size: 10000
19:30:09.197645 DEBUG [node] Connected block **[0000000000185113433faa1394dc69843c7dc9a1c2257dc48a993d9ec7389ea9] at height [834610]** from [47.74.21.139:18333] (1073743871, 4).
19:30:09.197666 INFO [blockchain] **Block [834610]    1 txs**    1 ins    0 wms    0 vms   41 vµs    9 rµs    9 cµs    7 pµs   13 aµs    1 sµs   53 dµs 0.000000
19:30:09.197696 VERBOSE [network] Received block from [47.74.21.139:18333] (212 bytes)
19:30:09.197892 DEBUG [node] Connected block [00000000003073ac377079246ef98551c236ad11fed13bbb69f7bd33a5929555] at height [834611] from [47.74.21.139:18333] (1073743871, 4).
19:30:09.197929 INFO [blockchain] **Block [834611]    1 txs**    1 ins    0 wms    0 vms   39 vµs    9 rµs   10 cµs    6 pµs   11 aµs    1 sµs   55 dµs 0.000000
19:30:09.197962 VERBOSE [network] Received block from [47.74.21.139:18333] (212 bytes)
19:30:09.198160 DEBUG [node] Connected block **[000000000028dd84d602c3288b08ba9ed9312f60d99e4342ab5f26c33f8dd699] at height [834612]** from [47.74.21.139:18333] (1073743871, 4).
19:30:09.198182 INFO [blockchain] **Block [834612]    1 txs**    1 ins    0 wms    0 vms   40 vµs    8 rµs   12 cµs    6 pµs   11 aµs    1 sµs   54 dµs 0.000000
19:30:09.198214 VERBOSE [network] Received block from [47.74.21.139:18333] (212 bytes)
19:30:09.198761 DEBUG [node] Connected block **[00000000003734667ba283abfe1597155e1d7a1e0a6d121ab18126dcdb3e3ed6] at height [834613]** from [47.74.21.139:18333] (1073743871, 4).
19:30:09.198783 INFO [blockchain] **Block [834613]    1 txs**    1 ins    0 wms    0 vms  365 vµs    8 rµs  329 cµs    9 pµs   15 aµs    1 sµs   65 dµs 0.000000
19:30:09.198818 VERBOSE [network] Received block from [47.74.21.139:18333] (212 bytes)
19:30:09.199015 DEBUG [node] Connected block **[00000000002280756eca2d74fa78fd899d284cb6c79fff296b5a1936f6877d3d] at height [834614]** from [47.74.21.139:18333] (1073743871, 4).
19:30:09.199034 INFO [blockchain] **Block [834614]    1 txs**    1 ins    0 wms    0 vms   36 vµs    9 rµs    8 cµs    6 pµs   10 aµs    1 sµs   47 dµs 0.000000
19:30:09.629737 VERBOSE [network] Received block from [47.74.21.139:18333] (212 bytes)
19:30:09.630044 DEBUG [node] Connected block **[000000000015f78cfbe617f7d410c89ba78f667b10c36e02a7602b76ac612546] at height [834615]** from [47.74.21.139:18333] (1073743871, 4).
19:30:09.630068 INFO [blockchain] **Block [834615]    1 txs**    1 ins    0 wms    0 vms   61 vµs   14 rµs   17 cµs   11 pµs   15 aµs    2 sµs   72 dµs 0.000000
19:30:09.630102 VERBOSE [network] Received block from [47.74.21.139:18333] (212 bytes)
19:30:09.630363 DEBUG [node] Connected block **[00000000002e736e6241d444d63e24bf291fc78862bbd7d800e03fd324ab1752] at height [834616]** from [47.74.21.139:18333] (1073743871, 4).
19:30:09.630385 INFO [blockchain] **Block [834616]    1 txs**    1 ins    0 wms    0 vms   64 vµs    9 rµs   24 cµs    6 pµs   23 aµs    1 sµs   76 dµs 0.000000
19:30:09.630415 VERBOSE [network] Received block from [47.74.21.139:18333] (212 bytes)
19:30:09.630688 DEBUG [node] Connected block **[000000000007efdf097b3b242986b6ecc903f27ebf0e37fe3debbc37ef079184] at height [834617]** from [47.74.21.139:18333] (1073743871, 4).
19:30:09.630707 INFO [blockchain] **Block [834617]    1 txs**    1 ins    0 wms    0 vms   75 vµs   21 rµs   21 cµs    6 pµs   24 aµs    1 sµs   69 dµs 0.000000
19:30:09.630734 VERBOSE [network] Received block from [47.74.21.139:18333] (212 bytes)
19:30:09.630967 DEBUG [node] Connected block **[000000000007455bb28f57909def88dcc7b481412f3a112fc69aac85fd247ed0] at height [834618]** from [47.74.21.139:18333] (1073743871, 4).
19:30:09.630988 INFO [blockchain] **Block [834618]    1 txs**    1 ins    0 wms    0 vms   67 vµs    8 rµs   23 cµs   21 pµs   12 aµs    1 sµs   51 dµs 0.000000
19:30:09.631019 VERBOSE [network] Received block from [47.74.21.139:18333] (212 bytes)
19:30:09.631220 DEBUG [node] Connected block **[00000000003c7e7471814815446168213734944e1df4c8304b08ec750042db19] at height [834619]** from [47.74.21.139:18333] (1073743871, 4).
19:30:09.631253 INFO [blockchain] **Block [834619]    1 txs**    1 ins    0 wms    0 vms   36 vµs    8 rµs    9 cµs    6 pµs   10 aµs    1 sµs   65 dµs 0.000000
19:30:09.631282 VERBOSE [network] Received block from [47.74.21.139:18333] (212 bytes)
19:30:09.631471 DEBUG [node] Connected block **[00000000001b3e4ad0b66a60d3d6ca15a986906fa22619d5152531017737264e] at height [834620]** from [47.74.21.139:18333] (1073743871, 4).
19:30:09.631492 INFO [blockchain] **Block [834620]    1 txs**    1 ins    0 wms    0 vms   40 vµs    7 rµs   12 cµs    6 pµs   12 aµs    1 sµs   49 dµs 0.000000
19:30:09.631521 VERBOSE [network] Received block from [47.74.21.139:18333] (212 bytes)
19:30:09.631768 DEBUG [node] Connected block **[000000000026de12904cf8a2230bee656475f0016ad7cb320fcba500e22c3d87] at height [834621]** from [47.74.21.139:18333] (1073743871, 4).
19:30:09.631787 INFO [blockchain] **Block [834621]    1 txs**    1 ins    0 wms    0 vms   42 vµs    9 rµs    9 cµs    7 pµs   14 aµs    1 sµs   58 dµs 0.000000
19:30:09.654093 VERBOSE [network] Received block from [47.74.21.139:18333] (212 bytes)
19:30:09.654483 DEBUG [node] Connected block **[000000000017b4fa42f27060f764a36a2e837fdb59771a6ba26d1b80e5925ed2] at height [834622]** from [47.74.21.139:18333] (1073743871, 4).
19:30:09.654507 INFO [blockchain] **Block [834622]    1 txs**    1 ins    0 wms    0 vms   70 vµs   13 rµs   16 cµs   10 pµs   26 aµs    2 sµs   68 dµs 0.000000
19:30:09.654543 VERBOSE [network] Received block from [47.74.21.139:18333] (211 bytes)
19:30:09.654878 DEBUG [node] Connected block **[0000000000067bd89409368d221507a160e5c45972eeb01efe210054fe8e7d85] at height [834623]** from [47.74.21.139:18333] (1073743871, 4).
19:30:09.654902 INFO [blockchain] **Block [834623]    1 txs**    1 ins    0 wms    0 vms   42 vµs   10 rµs   11 cµs    7 pµs   11 aµs    1 sµs  121 dµs 0.000000
19:30:09.654936 VERBOSE [network] Received block from [47.74.21.139:18333] (212 bytes)
19:30:09.655142 DEBUG [node] Connected block **[00000000002b980fcd729daaa248fd9316a5200e9b367f4ff2c42453e84201ca] at height [834624]** from [47.74.21.139:18333] (1073758207, 4).
19:30:09.655162 INFO [blockchain] **Block [834624]    1 txs**    1 ins    0 wms    0 vms   41 vµs    9 rµs   10 cµs    7 pµs   12 aµs    1 sµs   49 dµs 0.000000
19:30:09.655192 VERBOSE [network] Received block from [47.74.21.139:18333] (212 bytes)
19:30:09.655398 DEBUG [node] Connected block **[00000000003286d4e7e8f85e85d5352adba06123668f511bc02aece51efe29d8] at height [834625]** from [47.74.21.139:18333] (1073758207, 4).
19:30:09.655417 INFO [blockchain] **Block [834625]    1 txs**    1 ins    0 wms    0 vms   38 vµs    8 rµs    8 cµs    6 pµs   13 aµs    1 sµs   66 dµs 0.000000
19:30:09.655445 VERBOSE [network] Received block from [47.74.21.139:18333] (212 bytes)
19:30:09.655730 DEBUG [node] Connected block **[0000000000109c1c03d1acdaa0a7cb40e1cb68aad8016bf45bda2d24c3d9794a] at height [834626]** from [47.74.21.139:18333] (1073758207, 4).
19:30:09.655749 INFO [blockchain] **Block [834626]    1 txs**    1 ins    0 wms    0 vms   35 vµs    7 rµs    8 cµs    6 pµs   11 aµs    1 sµs   80 dµs 0.000000
19:30:09.655776 VERBOSE [network] Received block from [47.74.21.139:18333] (212 bytes)
19:30:09.656032 DEBUG [node] Connected block **[000000000010082700fc4e13839a95b6b0736dd225aa881c74b733bce8f23667] at height [834627]** from [47.74.21.139:18333] (1073758207, 4).
19:30:09.656050 INFO [blockchain] **Block [834627]    1 txs**    1 ins    0 wms    0 vms   34 vµs    8 rµs    8 cµs    6 pµs   11 aµs    1 sµs   90 dµs 0.000000
19:30:09.656078 VERBOSE [network] Received block from [47.74.21.139:18333] (212 bytes)
19:30:09.656342 DEBUG [node] Connected block [00000000002e9d771e65ce0ab5989c19c343b90a604e23de1d4647f2916f1acb] at height [834628] from [47.74.21.139:18333] (1073758207, 4).
19:30:09.656363 INFO [blockchain] Block [834628]    1 txs    1 ins    0 wms    0 vms   48 vµs   20 rµs    9 cµs    6 pµs   10 aµs    1 sµs   58 dµs 0.000000
19:30:09.656393 VERBOSE [network] Received block from [47.74.21.139:18333] (212 bytes)
19:30:09.656654 DEBUG [node] Connected block **[00000000002adf2e77753596e675561ee12350c8006c3e6d971ae9a60565283f] at height [834629]** from [47.74.21.139:18333] (1073758207, 4).
19:30:09.656674 INFO [blockchain] **Block [834629]    1 txs**    1 ins    0 wms    0 vms   37 vµs    8 rµs    9 cµs    6 pµs   11 aµs    1 sµs   87 dµs 0.000000
19:30:09.656702 VERBOSE [network] Received block from [47.74.21.139:18333] (212 bytes)
19:30:09.656946 DEBUG [node] Connected block **[000000000024802f79e59625ad41235bf7acdc8d37e2b78c75d24268984674fe] at height [834630]** from [47.74.21.139:18333] (1073758207, 4).
19:30:09.656964 INFO [blockchain] **Block [834630]    1 txs**    1 ins    0 wms    0 vms   35 vµs    8 rµs    8 cµs    6 pµs   10 aµs    1 sµs   78 dµs 0.000000
19:30:09.656992 VERBOSE [network] Received block from [47.74.21.139:18333] (211 bytes)
19:30:09.657318 DEBUG [node] Connected block **[00000000003cd590bb017198b4a5be7dddf35ef9178ac767fdd3de986e8a2be3] at height [834631]** from [47.74.21.139:18333] (1073758207, 4).
19:30:09.657349 INFO [blockchain] **Block [834631]    1 txs**    1 ins    0 wms    0 vms   80 vµs   23 rµs   23 cµs    6 pµs   25 aµs    1 sµs   87 dµs 0.000000
19:30:09.900318 VERBOSE [network] Received block from [216.66.17.57:18333] (212 bytes)
19:30:09.900555 DEBUG [node] Connected block **[000000000001a253d0289f6be918a3018b0b98da4f87bbae62223e613f69071b] at height [834632]** from [216.66.17.57:18333] (1073758207, 4).
19:30:09.900577 INFO [blockchain] **Block [834632]    1 txs**    1 ins    0 wms    0 vms   46 vµs    7 rµs   12 cµs   10 pµs   13 aµs    2 sµs   81 dµs 0.000000
19:30:10.107193 VERBOSE [network] Received block from [216.66.17.57:18333] (212 bytes)
19:30:10.107443 DEBUG [node] Connected block **[0000000000093579b68122a08a459be4a2837810a9088231e408b9f512c9ff6e] at height [834633]** from [216.66.17.57:18333] (1073758207, 4).
19:30:10.107464 INFO [blockchain] **Block [834633]    1 txs**    1 ins    0 wms    0 vms   51 vµs    8 rµs   12 cµs    9 pµs   18 aµs    2 sµs   78 dµs 0.000000
19:30:10.107495 VERBOSE [network] Received block from [216.66.17.57:18333] (212 bytes)
19:30:10.107742 DEBUG [node] Connected block **[000000000015c61cc1430d989ac68e88ead7b14f6240c7644f3b6ecb657aae05] at height [834634]** from [216.66.17.57:18333] (1073758207, 4).
19:30:10.111731 INFO [blockchain] **Block [834634]    1 txs**    1 ins    0 wms    0 vms   39 vµs    8 rµs   10 cµs    7 pµs   12 aµs    1 sµs   67 dµs 0.000000
19:30:10.112146 DEBUG [node] Connected block **[00000000000869293b106c35c90d91b3c6a4c09dc06090b1b96f4190eaae226c] at height [834635]** from [47.74.21.139:18333] (1073758207, 4).
19:30:10.112168 INFO [blockchain] **Block [834635]    1 txs**    1 ins    0 wms    0 vms   52 vµs   11 rµs   14 cµs    9 pµs   14 aµs    2 sµs   59 dµs 0.000000
19:30:10.112200 VERBOSE [network] Received block from [47.74.21.139:18333] (212 bytes)
19:30:10.112432 DEBUG [node] Connected block **[00000000002b6fe67d1807354e5c609413dbdaa701927bc7cf14d7f5e308523b] at height [834636]** from [47.74.21.139:18333] (1073758207, 4).
19:30:10.112452 INFO [blockchain] **Block [834636]    1 txs**    1 ins    0 wms    0 vms   44 vµs    9 rµs    8 cµs    9 pµs   16 aµs    1 sµs   49 dµs 0.000000
19:30:10.112482 VERBOSE [network] Received block from [47.74.21.139:18333] (212 bytes)
19:30:10.112797 DEBUG [node] Connected block **[000000000006ffd3fd03bc0ff0e7a9b2aa1c321491d2f845d6a7aaf1021ac026] at height [834637]** from [47.74.21.139:18333] (1073758207, 4).
19:30:10.112839 INFO [blockchain] **Block [834637]    1 txs**    1 ins    0 wms    0 vms   62 vµs   25 rµs   11 cµs    7 pµs   16 aµs    1 sµs   95 dµs 0.000000
19:30:10.112942 VERBOSE [network] Received block from [47.74.21.139:18333] (212 bytes)
19:30:10.113427 DEBUG [node] Connected block **[00000000000631828f8d0be17d3d9d65fca2e6fb0dae539f60548e2f11b78b3d] at height [834638]** from [47.74.21.139:18333] (1073758207, 4).
19:30:10.113450 INFO [blockchain] **Block [834638]    1 txs**    1 ins    0 wms    0 vms   69 vµs   10 rµs    9 cµs   19 pµs   26 aµs    1 sµs  109 dµs 0.000000
19:30:10.113494 VERBOSE [network] Received block from [47.74.21.139:18333] (212 bytes)
19:30:10.113776 DEBUG [node] Connected block **[00000000002eee463b529ccb755acd7338af32033d4ecd635eabf0863db7c89e] at height [834639]** from [47.74.21.139:18333] (1073758207, 4).
19:30:10.113818 INFO [blockchain] Block [834639]    1 txs    1 ins    0 wms    0 vms   68 vµs    9 rµs   10 cµs   20 pµs   27 aµs    1 sµs   64 dµs 0.000000
19:30:10.113879 VERBOSE [network] Received block from [47.74.21.139:18333] (212 bytes)
19:30:10.114192 DEBUG [node] Connected block **[00000000003e22702f0d8033a27e249804f7d3cb7c41158bcdeebd5b773f1097] at height [834640]** from [47.74.21.139:18333] (1073758207, 4).
19:30:10.114236 INFO [blockchain] **Block [834640]    1 txs**    1 ins    0 wms    0 vms   55 vµs    9 rµs   25 cµs    6 pµs   12 aµs    1 sµs  102 dµs 0.000000
19:30:10.114358 VERBOSE [network] Received block from [216.66.17.57:18333] (212 bytes)
19:30:10.114695 DEBUG [node] Connected block [0000000000244a1cc779ea361dfcacd0b6b870ae5bc966c8578dbb267cd7e95c] at height [834641] from [216.66.17.57:18333] (1073758207, 4).
19:30:10.114736 INFO [blockchain] **Block [834641]    1 txs**    1 ins    0 wms    0 vms   67 vµs    9 rµs   22 cµs    7 pµs   26 aµs    1 sµs  101 dµs 0.000000
19:30:10.114769 VERBOSE [network] Received block from [216.66.17.57:18333] (212 bytes)
19:30:10.114951 DEBUG [database] Output cache hit rate: 0.162298, size: 10000
19:30:10.115042 DEBUG [node] Connected block **[000000000026b7e99085355ebc9aa7e0c8cfeceee2189978e7146edd8de352c3] at height [834642]** from [216.66.17.57:18333] (1073758207, 4).
19:30:10.115062 INFO [blockchain] **Block [834642]    1 txs**    1 ins    0 wms    0 vms   45 vµs   12 rµs    9 cµs    7 pµs   13 aµs    1 sµs   95 dµs 0.000000
19:30:10.115124 VERBOSE [network] Received block from [47.74.21.139:18333] (212 bytes)
19:30:10.115342 DEBUG [node] Connected block **[00000000001b80dc232b9d396e4d31941f71a36f030f802ca018b70457719de5] at height [834643]** from [47.74.21.139:18333] (1073758207, 4).
19:30:10.115363 INFO [blockchain] **Block [834643]    1 txs**    1 ins    0 wms    0 vms   39 vµs    8 rµs   10 cµs    7 pµs   11 aµs    1 sµs   70 dµs 0.000000
19:30:10.555724 VERBOSE [network] Received block from [216.66.17.57:18333] (212 bytes)
19:30:10.555958 DEBUG [node] Connected block [000000000022a801c27e0058f34462f3a941b9d30272720445a5070741e6bb75] at height [834644] from [216.66.17.57:18333] (1073758207, 4).
19:30:10.555979 INFO [blockchain] **Block [834644]    1 txs**    1 ins    0 wms    0 vms   51 vµs    7 rµs    8 cµs   10 pµs   23 aµs    2 sµs   58 dµs 0.000000
19:30:10.556011 VERBOSE [network] Received block from [216.66.17.57:18333] (212 bytes)
19:30:10.556209 DEBUG [node] Connected block **[00000000002b697ea8976f34158b4bef351b75e5701dcfe65cffd4e0b1d2658a] at height [834645]** from [216.66.17.57:18333] (1073758207, 4).
19:30:10.556227 INFO [blockchain] **Block [834645]    1 txs**    1 ins    0 wms    0 vms   37 vµs    7 rµs   10 cµs    6 pµs   11 aµs    1 sµs   53 dµs 0.000000
19:30:10.556256 VERBOSE [network] Received block from [216.66.17.57:18333] (212 bytes)
19:30:10.556545 DEBUG [node] Connected block **[0000000000382f3c9441c00ceac7aeedf053c0b82669749d2adcb835b936b601] at height [834646]** from [216.66.17.57:18333] (1073758207, 4).
19:30:10.556568 INFO [blockchain] **Block [834646]    1 txs**    1 ins    0 wms    0 vms  104 vµs    8 rµs    8 cµs    6 pµs   80 aµs    1 sµs   59 dµs 0.000000
19:30:10.556600 VERBOSE [network] Received block from [216.66.17.57:18333] (212 bytes)
19:30:10.556890 DEBUG [node] Connected block **[00000000003fe28ac7bb359b2eee508f718d3011dc1eb0be79be145060796814] at height [834647]** from [216.66.17.57:18333] (1073758207, 4).
19:30:10.556911 INFO [blockchain] **Block [834647]    1 txs**    1 ins    0 wms    0 vms   56 vµs    8 rµs    9 cµs    7 pµs   28 aµs    1 sµs   71 dµs 0.000000
19:30:10.556941 VERBOSE [network] Received block from [216.66.17.57:18333] (212 bytes)
19:30:10.557151 DEBUG [node] Connected block [0000000000227bce5756077b59dbe545e1d211f0f7aa5934d81eefe979867d5b] at height [834648] from [216.66.17.57:18333] (1073758207, 4).
19:30:10.557172 INFO [blockchain] Block [834648]    1 txs    1 ins    0 wms    0 vms   42 vµs    9 rµs    9 cµs    6 pµs   15 aµs    1 sµs   52 dµs 0.000000
19:30:10.557214 VERBOSE [network] Received block from [216.66.17.57:18333] (212 bytes)
19:30:10.557543 DEBUG [node] Connected block **[00000000002d1f9bcde4d5780e6cbfe84dd1b63dac8e4f0fa4b3c490757b9a3d] at height [834649]** from [216.66.17.57:18333] (1073758207, 4).
19:30:10.557593 INFO [blockchain] **Block [834649]    1 txs**    1 ins    0 wms    0 vms   54 vµs    8 rµs   24 cµs    7 pµs   12 aµs    1 sµs   72 dµs 0.000000
19:30:10.557633 VERBOSE [network] Received block from [216.66.17.57:18333] (212 bytes)
19:30:10.557742 VERBOSE [network] Received block from [47.74.21.139:18333] (212 bytes)
19:30:10.557828 DEBUG [database] Output cache hit rate: 0.162298, size: 10000
19:30:10.557905 DEBUG [node] Connected block [000000000028100732f4d01dcc1abef88c5081f355abdb9fdaa994a59b6fe7e5] at height [834650] from [216.66.17.57:18333] (1073758207, 4).
19:30:10.557947 INFO [blockchain] Block [834650]    1 txs    1 ins    0 wms    0 vms   72 vµs   12 rµs   10 cµs   20 pµs   26 aµs    1 sµs   52 dµs 0.000000
19:30:10.558072 VERBOSE [network] Received block from [47.74.21.139:18333] (212 bytes)
19:30:10.558210 DEBUG [database] Output cache hit rate: 0.162298, size: 10000
19:30:10.558275 DEBUG [node] Connected block [00000000003ef975510a4b16fc62080754cece40eb9479b6a62f82f736099e2d] at height [834651] from [216.66.17.57:18333] (1073758207, 4).
19:30:10.558319 INFO [blockchain] Block [834651]    1 txs    1 ins    0 wms    0 vms   67 vµs   37 rµs    9 cµs    6 pµs   12 aµs    1 sµs  100 dµs 0.000000
19:30:10.558474 VERBOSE [network] Received block from [47.74.21.139:18333] (212 bytes)
19:30:10.558718 DEBUG [node] Connected block **[00000000003ee4b405223b63a5a92ee8e64762e630e663b7a80a8c1e49d04296] at height [834652]** from [216.66.17.57:18333] (1073758207, 4).
19:30:10.558762 INFO [blockchain] **Block [834652]    1 txs**    1 ins    0 wms    0 vms   74 vµs    8 rµs   24 cµs   22 pµs   16 aµs    1 sµs  100 dµs 0.000000
19:30:10.559198 VERBOSE [network] Received block from [47.74.21.139:18333] (212 bytes)
19:30:10.559533 DEBUG [node] Connected block **[000000000005ac470e88fe29eaa344cb06376cba353c6fc12da1038791f9d230] at height [834653]** from [47.74.21.139:18333] (1073758207, 4).
19:30:10.559554 INFO [blockchain] **Block [834653]    1 txs**    1 ins    0 wms    0 vms   52 vµs   22 rµs    9 cµs    6 pµs   11 aµs    1 sµs   82 dµs 0.000000
19:30:10.572762 VERBOSE [network] Received block from [216.66.17.57:18333] (212 bytes)
19:30:10.572980 DEBUG [node] Connected block **[000000000039f2af84300fbe532632384d824d7ee8205f4fc9d5b8550c351666] at height [834654]** from [216.66.17.57:18333] (1073758207, 4).
19:30:10.573000 INFO [blockchain] **Block [834654]    1 txs**    1 ins    0 wms    0 vms   42 vµs    7 rµs    9 cµs    9 pµs   13 aµs    2 sµs   59 dµs 0.000000
19:30:10.573041 VERBOSE [network] Received block from [216.66.17.57:18333] (212 bytes)
19:30:10.573347 DEBUG [node] Connected block **[0000000000037356d39f5537ce74f6f3f88175295fc56e49141a87ace5d84740] at height [834655]** from [216.66.17.57:18333] (1073758207, 4).
19:30:10.573369 INFO [blockchain] **Block [834655]    1 txs**    1 ins    0 wms    0 vms   42 vµs   10 rµs    9 cµs    7 pµs   12 aµs    1 sµs   64 dµs 0.000000
19:30:10.573401 VERBOSE [network] Received block from [216.66.17.57:18333] (212 bytes)
19:30:10.573661 DEBUG [node] Connected block **[00000000001ed7e30f7a3ba28e8cdfd26c25f4890216db49a3eb12c08e2e229b] at height [834656]** from [216.66.17.57:18333] (1073758207, 4).
19:30:10.573703 INFO [blockchain] **Block [834656]    1 txs**    1 ins    0 wms    0 vms   67 vµs    7 rµs   22 cµs    6 pµs   28 aµs    1 sµs   61 dµs 0.000000
19:30:10.573759 VERBOSE [network] Received block from [216.66.17.57:18333] (212 bytes)
19:30:10.574074 DEBUG [node] Connected block **[0000000000134dfb1b3688b2f7944d43daa1356ff2a47a280a1c6b254a84538a] at height [834657]** from [216.66.17.57:18333] (1073758207, 4).
19:30:10.574118 INFO [blockchain] Block [834657]    1 txs    1 ins    0 wms    0 vms   82 vµs   21 rµs   22 cµs   21 pµs   15 aµs    1 sµs  106 dµs 0.000000
19:30:10.574180 VERBOSE [network] Received block from [216.66.17.57:18333] (212 bytes)
19:30:10.574460 DEBUG [node] Connected block **[00000000003b5b7ef5e5d5944f2b0ec688884cbef8c22b3fa76a882b01d370d2] at height [834658]** from [216.66.17.57:18333] (1073758207, 4).
19:30:10.574502 INFO [blockchain] **Block [834658]    1 txs**    1 ins    0 wms    0 vms   67 vµs   23 rµs   24 cµs    6 pµs   11 aµs    1 sµs   86 dµs 0.000000
19:30:10.574543 VERBOSE [network] Received block from [216.66.17.57:18333] (212 bytes)
19:30:10.574857 DEBUG [node] Connected block **[00000000001a32a596a2919566741c6b2a9bd5ea57fb6ea3aeb3e08516360e6a] at height [834659]** from [216.66.17.57:18333] (1073758207, 4).
19:30:10.574915 INFO [blockchain] **Block [834659]    1 txs**    1 ins    0 wms    0 vms   62 vµs    7 rµs   22 cµs   19 pµs   11 aµs    1 sµs   91 dµs 0.000000
19:30:10.574986 VERBOSE [network] Received block from [216.66.17.57:18333] (212 bytes)
19:30:10.575398 DEBUG [node] Connected block **[00000000003b46a92d281d66f72a39cf17c9dd0c1ae2d5dac0a99f1ff2c707fc] at height [834660]** from [216.66.17.57:18333] (1073758207, 4).
19:30:10.575443 INFO [blockchain] **Block [834660]    1 txs**    1 ins    0 wms    0 vms   77 vµs    8 rµs    9 cµs   19 pµs   26 aµs   14 sµs  103 dµs 0.000000
19:30:10.575473 VERBOSE [network] Received block from [216.66.17.57:18333] (212 bytes)
19:30:10.575690 DEBUG [database] Output cache hit rate: 0.162298, size: 10000
19:30:10.575747 DEBUG [node] Connected block **[00000000001ba7f60688f19dcf61289d7ca24f7fc9ba18ec15ef1b59efe747f2] at height [834661]** from [216.66.17.57:18333] (1073758207, 4).
19:30:10.575794 INFO [blockchain] **Block [834661]    1 txs**    1 ins    0 wms    0 vms   60 vµs   21 rµs   21 cµs    6 pµs   10 aµs    1 sµs   90 dµs 0.000000
19:30:10.575849 VERBOSE [network] Received block from [47.74.21.139:18333] (212 bytes)
19:30:10.576186 DEBUG [node] Connected block **[00000000002e338ac959e7719f58a067c57058958de065a50a39bd194aa2c7e2] at height [834662]** from [47.74.21.139:18333] (1073758207, 4).
19:30:10.576207 INFO [blockchain] **Block [834662]    1 txs**    1 ins    0 wms    0 vms   82 vµs   23 rµs    8 cµs    6 pµs   43 aµs    1 sµs  118 dµs 0.000000
19:30:11.004110 VERBOSE [network] Received block from [216.66.17.57:18333] (212 bytes)
19:30:11.004261 VERBOSE [system] 0x70000e1e9000 script::verify()
19:30:11.004287 VERBOSE [system] 0x70000e1e9000 script::is_pay_to_witness bien
19:30:11.004296 VERBOSE [system] 0x70000e1e9000 script::is_pay_to_witness biwi is false
19:30:11.004304 VERBOSE [system] 0x70000e1e9000 script::verify() is_pay_to_script_hash()
19:30:11.004606 VERBOSE [system] 0x70000e1e9000 script::is_pay_to_witness bien
19:30:11.004748 VERBOSE [system] 0x70000e1e9000 script::is_pay_to_witness biwi is false
19:30:11.005664 DEBUG [node] Connected block **[00000000001e369d7ecc61a7eaa89bfc6c39636f31af9dc607c78a091755287b] at height [834663]** from [216.66.17.57:18333] (1073758207, 4).
19:30:11.005694 INFO [blockchain] **Block [834663]    2 txs**    1 ins    0 wms    1 vms  586 vµs   13 rµs   15 cµs   21 pµs   22 aµs  513 sµs  407 dµs 0.000000

JasonCoombs commented 6 years ago

here's what my v4 log looks like when validating the block that should fail (which does fail validation with error: invalid_witness in my development branch where I have ensured segwit activates correctly):

block 872730 fails validation in v4 if segwit is activated https://github.com/libbitcoin/libbitcoin-node/issues/381

Note: it is the next-to-last tx (44 of 45, index number 43 because the tx numbering is zero-based) in block 872,730 which is the tx (the first one ever on testnet?) that mixes segwit and non-segwit input

21:33:17.476781 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.476788 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi

see: https://drive.google.com/file/d/0B7E799YMGCWTc3BUNFNndUVLS2c/view

this is tx id 118c83b2c7aa4b500e0c0333dbfef990ca6c324004d68f68acb10ddada918474 as shown in the additional debug output, below: https://github.com/libbitcoin/libbitcoin-node/issues/383#issuecomment-427628654

looking up the block and tx index numbers using bx, we see:

$ bx fetch-tx-index 118c83b2c7aa4b500e0c0333dbfef990ca6c324004d68f68acb10ddada918474
metadata
{
    hash 118c83b2c7aa4b500e0c0333dbfef990ca6c324004d68f68acb10ddada918474
    height 872730
    index 43
}

And in hex: https://testnet.blockchain.info/tx/118c83b2c7aa4b500e0c0333dbfef990ca6c324004d68f68acb10ddada918474?format=hex

because is_enabled(forks, rule_fork::bip141_rule); incorrectly returns false in v4, the bug is bypassed and we're told the block successfully validated. validation fails currently when it is attempted properly. each log output indicating script::is_pay_to_witness mal should instead show us script::is_pay_to_witness bien as v3 does in the log shown above when using the same additional debug logging added to script.cpp at the top of this Issue.

21:33:17.334122 DEBUG [blockchain] Validated block #872729 [0000000000000a17e01273a1399d3bcf53f1dbbe2a799b43d4a46538b8ae97a2]
21:33:17.334197 DEBUG [database] Output cache hit rate: 0.446848, size: 4306
21:33:17.336309 INFO [blockchain] Organized blocks [872729-872729]
21:33:17.367798 DEBUG [blockchain] Got next block #872730
21:33:17.470773 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.470929 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.470942 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.470950 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.470958 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.471071 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.471089 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.471097 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.471105 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.471199 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.471208 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.471215 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.471222 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.471316 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.471324 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.471331 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.471337 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.471431 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.471440 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.471446 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.471453 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.471546 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.471554 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.471561 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.471568 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.471661 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.471670 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.471676 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.471683 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.471776 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.471785 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.471791 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.471798 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.471893 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.471902 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.471908 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.471915 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.472007 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.472016 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.472022 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.472029 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.472125 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.472134 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.472140 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.472147 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.472243 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.472251 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.472258 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.472264 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.472357 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.472365 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.472372 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.472378 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.472472 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.472481 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.472487 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.472498 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.472593 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.472601 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.472608 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.472614 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.472710 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.472719 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.472725 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.472732 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.472824 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.472833 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.472839 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.472846 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.472940 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.472949 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.472955 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.472962 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.473055 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.473063 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.473070 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.473076 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.473172 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.473180 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.473187 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.473194 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.473287 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.473295 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.473302 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.473308 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.473404 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.473412 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.473419 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.473426 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.473517 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.473526 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.473532 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.473539 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.473632 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.473641 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.473647 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.473654 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.473747 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.473755 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.473762 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.473768 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.473862 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.473870 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.473877 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.473883 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.473979 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.473988 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.473994 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.474001 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.474096 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.474105 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.474111 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.474118 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.474210 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.474219 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.474225 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.474232 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.474327 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.474336 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.474342 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.474349 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.474443 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.474452 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.474459 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.474465 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.474560 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.474569 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.474575 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.474582 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.474675 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.474683 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.474705 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.474711 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.474837 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.474846 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.474853 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.474860 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.474961 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.474971 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.474977 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.474984 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.475082 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.475091 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.475097 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.475104 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.475202 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.475211 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.475218 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.475225 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.475324 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.475333 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.475340 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.475347 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.475457 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.475466 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.475479 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.475487 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.475501 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.475508 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.475782 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.475794 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.475802 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.475809 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.475829 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.475837 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.476105 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.476116 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.476123 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.476131 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.476238 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.476247 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.476254 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.476261 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.476364 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.476373 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.476380 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.476387 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.476488 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.476497 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.476504 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.476511 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.476612 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.476621 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.476628 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.476635 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.476730 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.476739 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.476746 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.476753 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.476765 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.476772 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.476781 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.476788 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi
21:33:17.476796 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.476802 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.476903 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.476912 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.476919 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.476926 VERBOSE [system] 0x700005ee1000 script::verify()
21:33:17.477038 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness mal
21:33:17.477047 VERBOSE [system] 0x700005ee1000 script::is_pay_to_witness biwi is false
21:33:17.477054 VERBOSE [system] 0x700005ee1000 script::verify() finished, returning error::success
21:33:17.477278 DEBUG [blockchain] Validated block #872730 [0000000000000521adc862e31310e79d7f98455933833ba2c95d5e4b257a6aef]
21:33:17.477323 DEBUG [database] Output cache hit rate: 0.446602, size: 4352
21:33:17.478469 INFO [blockchain] Organized blocks [872730-872730]
21:33:17.500661 DEBUG [blockchain] Got next block #872731
21:33:18.713432 VERBOSE [system] 0x700005ee1000 script::verify()

JasonCoombs commented 6 years ago

Attempting to extract the new [bitcoin] settings section (with values for Testnet) from the settings.cpp file, as required by the new v4 config file, see: https://github.com/libbitcoin/libbitcoin-node/pull/384

JasonCoombs commented 6 years ago

With the new [bitcoin] settings in config, and Testnet selected for all, the invalid_witness error appears just as expected based on my testing in my development fork. There remains a validation problem with Segwit in v4, as detailed here https://github.com/libbitcoin/libbitcoin-node/issues/381

Now the block 872730 validation error should be easy to reproduce. Any testing with v4 previously would not have encountered this problem, because the old v3 method of activating bip141 using the bip141 config setting is no longer sufficient in v4.

06:02:39.843320 VERBOSE [system] 0x7000055ee000 script::is_pay_to_witness bien
06:02:39.843327 VERBOSE [system] 0x7000055ee000 script::is_pay_to_witness biwi
06:02:39.843334 VERBOSE [system] 0x7000055ee000 witness::verify() script_version::zero
06:02:39.843341 VERBOSE [system] 0x7000055ee000 witness::extract_embedded_script() out_stack.size != 2 || !is_push_size
06:02:39.843371 DEBUG [blockchain] Verify failed [872730] : invalid witness
 libconsensus : false
 forks        : 16382
 outpoint     : 1b9e1323923508ecc3d097f11aeaa66acbe6a6d3c4ea6b25d57d34ee9c8d660d:0
 script       : a914e5a679f81579cbf20bb2a4764be9e95e8b3e396787
 value        : 100000000
 inpoint      : 118c83b2c7aa4b500e0c0333dbfef990ca6c324004d68f68acb10ddada918474:0
 transaction  : 01000000020d668d9cee347dd5256beac4d3a6e6cb6aa6ea1af197d0c3ec08359223139e1b00000000171600143f0402fcb8beef28a89be7e94794ec66d469fdb6feffffff0d668d9cee347dd5256beac4d3a6e6cb6aa6ea1af197d0c3ec08359223139e1b010000006b483045022100eaceee00202693d7d54aa84bd8b216f62b1f041b16e08e0352eb1ed50f1ef10102201a90288f667f975cc8d524431d61bd8b15f5d189610d31a9bd12ee37c0674f06012103c4561ce27291b1730e5a429934f45fb4ff9b56e24b6e10e0d176f198fa029671feffffff02506c9800000000001976a9140af575373dad17150d91b6a191a371b59e09d87f88ac001c4e0e0000000017a91403572c975aec5228c0e2982ef8a03b6830e0554f8719510d00
06:02:43.767375 FATAL [blockchain] Failure in block organization, store is now corrupt: operation failed

JasonCoombs commented 6 years ago

The witness is missing from the transaction at least as early in the tx validation processing as when the transaction object is constructed, apparently from the data store because in v4 we store the txs prior to validation. It looks like the wire deserialization and storage to the database excludes witness data.

JasonCoombs commented 6 years ago

Using v3 the transaction at tx index 43 in block 872730 looks correct:

19:54:21.872358 VERBOSE [blockchain]  validate_block::connect_inputs() validating tx index 42 in Block 0000000000000521adc862e31310e79d7f98455933833ba2c95d5e4b257a6aef
 the tx is: 010000000001020d668d9cee347dd5256beac4d3a6e6cb6aa6ea1af197d0c3ec08359223139e1b00000000171600143f0402fcb8beef28a89be7e94794ec66d469fdb6feffffff0d668d9cee347dd5256beac4d3a6e6cb6aa6ea1af197d0c3ec08359223139e1b010000006b483045022100eaceee00202693d7d54aa84bd8b216f62b1f041b16e08e0352eb1ed50f1ef10102201a90288f667f975cc8d524431d61bd8b15f5d189610d31a9bd12ee37c0674f06012103c4561ce27291b1730e5a429934f45fb4ff9b56e24b6e10e0d176f198fa029671feffffff02506c9800000000001976a9140af575373dad17150d91b6a191a371b59e09d87f88ac001c4e0e0000000017a91403572c975aec5228c0e2982ef8a03b6830e0554f8702483045022100913d331b78a2c1a2ec0e2af7a826181f26e15e0195cef13f492d2e22b8dd9e2c02201f808621f9684574cbbd755933509f6bce7536aa5c36dd4c57fd635e54854049012103e2728bdc007032f5b30c823f4a3cd9236eab5e3a6f23c6ae6be8bd63ff8479220019510d00

v4 is missing these bytes near the end:

02483045022100913d331b78a2c1a2ec0e2af7a826181f26e15e0195cef13f492d2e22b8dd9e2c02201f808621f9684574cbbd755933509f6bce7536aa5c36dd4c57fd635e54854049012103e2728bdc007032f5b30c823f4a3cd9236eab5e3a6f23c6ae6be8bd63ff84792200

which v3 correctly parses into the following witness data:

19:54:21.872959 VERBOSE [system] 0x70000c359000 script::verify() input_witness = [3045022100913d331b78a2c1a2ec0e2af7a826181f26e15e0195cef13f492d2e22b8dd9e2c02201f808621f9684574cbbd755933509f6bce7536aa5c36dd4c57fd635e5485404901] [03e2728bdc007032f5b30c823f4a3cd9236eab5e3a6f23c6ae6be8bd63ff847922]

JasonCoombs commented 6 years ago

The problem could be here:

https://github.com/libbitcoin/libbitcoin/blob/master/src/message/block_transactions.cpp#L129

With v3 we appear to have used a different code path to get transactions into validation, and it seems likely that in v4 we now must store the witness at message processing time or it's gone forever. Still not seeing exactly how v3 ever got this right. Perhaps v3 has a bug that is only encountered if validation doesn't occur before shutdown? Maybe v3 forces validation of all pending transactions before it will shut down, because it was not designed to come back later and read witness data from the database in order to finish validation?

evoskuil commented 6 years ago

V3 validates blocks before storing and we have no reason to believe there is any problem with its validation. V4 validates blocks after storing (which follows PoW validation of current strong chain). V4 is incomplete.

JasonCoombs commented 6 years ago

v3 never wrote witness data to the transaction database because validation was always done before the transaction was written to storage. This:

https://github.com/libbitcoin/libbitcoin-database/blob/version3/src/databases/transaction_database.cpp#L223

and this:

https://github.com/libbitcoin/libbitcoin-database/blob/version3/src/databases/transaction_database.cpp#L260

        // WRITE THE TX
        tx.to_data(serial, false);

passing false in the wire parameter leaves the witness parameter to default (false) value also.

https://github.com/libbitcoin/libbitcoin/blob/master/include/bitcoin/bitcoin/chain/transaction.hpp#L131

I wonder whether these defaults should be changed to witness=true in v4. Also, there's going to need to be a conditional choice made as to whether to write witness data to the table at all. Bitcoin Cash and other forks will never include segwit so we don't need to write and read witnesses depending on forks.

evoskuil commented 6 years ago

Incorrect, see: https://github.com/libbitcoin/libbitcoin/blob/master/src/chain/input.cpp#L238

evoskuil commented 6 years ago

We always write the witness. This allows us to read it independent of chain state (such as soft forks).

JasonCoombs commented 6 years ago

Thanks for the link. So it makes no difference whatsoever if transaction.to_data() is passed a false value for the witness parameter, witness data is supposed to be stored in the transaction database anyway (in v4) and this is done at the inputs level by checking for the wire=false condition.

https://github.com/libbitcoin/libbitcoin/blob/master/src/chain/transaction.cpp#L461

I see now that inputs_ looks for !wire to write witness data, but that will only happen if an input has a non-empty() witness. This won't result in any witness data being written at all, ever, in the case of Bitcoin Cash, correct?

And the part that you were saying was incorrect about my summary above https://github.com/libbitcoin/libbitcoin-node/issues/383#issuecomment-431535173 was only the idea that we would want to make this conditional on forks activating segwit, right?

evoskuil commented 6 years ago

Can’t write data that doesn’t exist. Incorrect was that we aren’t storing the witness. The witness parameter is honored for wire serialization.

JasonCoombs commented 6 years ago

We don't end up writing unnecessary segwit markers or null length values by always attempting to write witness data from the inputs regardless of forks, presumably.

JasonCoombs commented 6 years ago

This was confusing, it looked like the fix might be to change this line from && to ||

if (witness && !wire)

But witness=false being passed in as a parameter is irrelevant because witness is reset to true by line 238 as you showed above.

evoskuil commented 6 years ago

Fix for what?

JasonCoombs commented 6 years ago

There is a bug in v4 code that I'm working to fix. It fails to handle witness data correctly in at least the one circumstance present in block 872730 on Testnet, when there's a mix of segwit and non-segwit inputs of the type present in that block. Apparently the witness data is either not being written to the database (which would be explained by a mixup in the witness flag passed in to to_data() but as you noted that is irrelevant as long as the inputs_ object is actually being told to write its content to store because the witness flag is changed if !wire condition is true) or else the bug is somewhere other than in the database writer.

As shown in the debugging logs and other analysis, above, there is no doubt that the witness data is missing when validation occurs. And I've conclusively determined that the transaction object constructor is called (via the move construction, FYI) and as early as that happening there is no witness data present in the inputs_ elements. It's my understanding that this move construction which creates the new transaction object is happening when the validation step retrieves the candidate block from the block (and transactions) tables where it has been stored previously and is awaiting validation.

JasonCoombs commented 6 years ago

We don't end up writing unnecessary segwit markers or null length values by always attempting to write witness data from the inputs regardless of forks, presumably.

but wouldn't it be wrong for any developer in the future to ever call to_data() and pass witness=true as a parameter if the forks don't support segwit? right now I don't see how our code would catch that condition, I think we would spit out the requested segwit marker and witness item count fields but leave witness empty. almost exactly the condition that I am seeing in the validation failure on block 872730.

JasonCoombs commented 6 years ago

Can’t write data that doesn’t exist. Incorrect was that we aren’t storing the witness. The witness parameter is honored for wire serialization.

You mean in v4, right? v3 never stored any witness data. Now in v4 we're supposed to store it, but in the case of block 872730 on Testnet that may not be happening. And this is the very first block ever with a transaction that attempts to use a segwit UTXO as an input, apparently, so if this one doesn't validate then I'm pretty sure no other blocks with transactions that use segwit UTXOs will, either. A working hypothesis is that the witness data is not present in the transaction table and that's why it's missing when it's needed for validation. I'm in the process now of confirming that the witness data is actually being written as intended.

JasonCoombs commented 6 years ago

We always write the witness. This allows us to read it independent of chain state (such as soft forks).

This makes sense. I'm glad that the witness data is going to be in the transaction table going forward. There are many situations in which it may prove useful and important in addition to soft forks or other future reorganizations. Is it your intention to enable one running instance of libbitcoin-node or -server to represent simultaneously (on-demand) parallel "views" of almost any future fork that complies with published bip protocol?

evoskuil commented 6 years ago

Is it your intention to enable one running instance of libbitcoin-node or -server to represent simultaneously (on-demand) parallel "views" of almost any future fork...

No

evoskuil commented 6 years ago

You mean in v4, right? v3 never stored any witness data.

V3 stores witness.

evoskuil commented 6 years ago

but wouldn't it be wrong for any developer in the future to ever call to_data() and pass witness=true as a parameter if the forks don't support segwit?

No, the parameterization is how chain state is provided (by the developer).

evoskuil commented 6 years ago

...almost exactly the condition that I am seeing in the validation failure on block 872730.

The condition "almost exactly" is the same as "not".

JasonCoombs commented 6 years ago

here is what v4 appears to be storing to the transaction table when storize is called for tx 44 in block 872730:

20:57:12.276056 VERBOSE [system]  tx hash: 118c83b2c7aa4b500e0c0333dbfef990ca6c324004d68f68acb10ddada918474 storize(): 0200ffffffff506c9800000000001976a9140af575373dad17150d91b6a191a371b59e09d87f88ac00ffffffff001c4e0e0000000017a91403572c975aec5228c0e2982ef8a03b6830e0554f87020d668d9cee347dd5256beac4d3a6e6cb6aa6ea1af197d0c3ec08359223139e1b0000171600143f0402fcb8beef28a89be7e94794ec66d469fdb600feffffff0d668d9cee347dd5256beac4d3a6e6cb6aa6ea1af197d0c3ec08359223139e1b01006b483045022100eaceee00202693d7d54aa84bd8b216f62b1f041b16e08e0352eb1ed50f1ef10102201a90288f667f975cc8d524431d61bd8b15f5d189610d31a9bd12ee37c0674f06012103c4561ce27291b1730e5a429934f45fb4ff9b56e24b6e10e0d176f198fa02967100fefffffffe19510d0001

comparing to blockchain info:

https://testnet.blockchain.info/tx/118c83b2c7aa4b500e0c0333dbfef990ca6c324004d68f68acb10ddada918474?format=hex

JasonCoombs commented 6 years ago

...almost exactly the condition that I am seeing in the validation failure on block 872730.

The condition "almost exactly" is the same as "not".

are you seriously going to criticize my word choices for as long as I attempt to contribute to this project?

evoskuil commented 6 years ago

It’s not a question of word choice. If the behaviors aren’t the same they are different.

JasonCoombs commented 6 years ago

it's a clue to follow when trying to figure out where the error might be.

evoskuil commented 6 years ago

And my observation is a caution about assumptions, not a nit on word choice.

JasonCoombs commented 6 years ago

please show me where v3 stores witness data, because I'm not seeing it. the following line results in the non-wire, non-witness tx data being written to the table:

https://github.com/libbitcoin/libbitcoin-database/blob/version3/src/databases/transaction_database.cpp#L261

as I pointed out three days ago https://github.com/libbitcoin/libbitcoin-node/issues/383#issuecomment-431535173 when using words you thought were incorrect.

please do correct my incorrect understanding, but could you do it with references to source code instead of from your memory and with rapid-fire reply quips? thank you.

JasonCoombs commented 6 years ago

are you saying that v3 does the same as v4, by ignoring the witness=false value and forcing it to true, here, if wire=false ?

https://github.com/libbitcoin/libbitcoin/blob/version3/src/chain/input.cpp

JasonCoombs commented 6 years ago

here is what v3 writes to the transaction table for tx 44 in block 872730, and it does appear to include the witness data, unlike in v4:

02:40:43.885564 DEBUG [database]  tx.hash(): 118c83b2c7aa4b500e0c0333dbfef990ca6c324004d68f68acb10ddada918474 transaction_database::store() tx: 02ffffffff506c9800000000001976a9140af575373dad17150d91b6a191a371b59e09d87f88acffffffff001c4e0e0000000017a91403572c975aec5228c0e2982ef8a03b6830e0554f87020d668d9cee347dd5256beac4d3a6e6cb6aa6ea1af197d0c3ec08359223139e1b0000171600143f0402fcb8beef28a89be7e94794ec66d469fdb602483045022100913d331b78a2c1a2ec0e2af7a826181f26e15e0195cef13f492d2e22b8dd9e2c02201f808621f9684574cbbd755933509f6bce7536aa5c36dd4c57fd635e54854049012103e2728bdc007032f5b30c823f4a3cd9236eab5e3a6f23c6ae6be8bd63ff847922feffffff0d668d9cee347dd5256beac4d3a6e6cb6aa6ea1af197d0c3ec08359223139e1b01006b483045022100eaceee00202693d7d54aa84bd8b216f62b1f041b16e08e0352eb1ed50f1ef10102201a90288f667f975cc8d524431d61bd8b15f5d189610d31a9bd12ee37c0674f06012103c4561ce27291b1730e5a429934f45fb4ff9b56e24b6e10e0d176f198fa02967100fefffffffe19510d0001

JasonCoombs commented 6 years ago

v3 does this to attach protocol_block_in to the p2p session (inbound/outbound):

https://github.com/libbitcoin/libbitcoin-node/blob/version3/src/sessions/session_inbound.cpp#L57 https://github.com/libbitcoin/libbitcoin-node/blob/version3/src/sessions/session_outbound.cpp#L57

v4 does this to attach protocol_block_sync to the p2p sessions instead:

https://github.com/libbitcoin/libbitcoin-node/blob/master/src/sessions/session_inbound.cpp#L61 https://github.com/libbitcoin/libbitcoin-node/blob/master/src/sessions/session_outbound.cpp#L60

in both v3 and v4 there is an extra step processing incoming blocks that might not include witnesses:

https://github.com/libbitcoin/libbitcoin-node/blob/master/src/protocols/protocol_block_in.cpp#L89

it looks like protocol_block_sync is not requesting witness data, or isn't receiving it from the peer.

JasonCoombs commented 6 years ago

the result of invoking protocol_block_sync to receive block 872730 from the peer is shown here, and transaction 44 is missing its witness data as early in the process as the call to handle_receive_block:

17:49:41.534392 DEBUG [blockchain] 0x70000f39d000 Got next block #872730
17:49:41.534564 DEBUG [blockchain] 0x70000f39d000 block_chain::organize() header hash: 00000000000001ecda56ccf87d6c158b471853f5a86c1fe530690cd0249216a9
17:49:41.542004 DEBUG [node] 0x70000f39d000 protocol_block_sync::handle_receive_block() calling reservation_->import() at height: 872730
17:49:41.542063 DEBUG [node] 0x70000f39d000 reservation::import() calling chain.organize() at height: 872730 for block: 00000020a297aeb83865a4d4439b792abedbf153cf3b9d39a17312e0170a000000000000b72796785e6b99195e1717d15eaea41380dd74642a8e0ab828aa8edde65d3f35c99a6d57d8ea0e1a69f0069e2d01000000010000000000000000000000000000000000000000000000000000000000000000ffffffff2f031a510d0004c99a6d5704b5ce12180cc48f6c57ef1b0f00000000000a636b706f6f6c0b2f416e7453747265616d2fffffffff02672fa812000000001976a914876fbb82ec05caa6af7a3b5e5a983aae6c6cc6d688ac0000000000000000266a24aa21a9edc70ff34ff0984d969c609f88d0dedf2c4489a50d487648ab01aff22bfc20bc7500000000010000000149f2d3e0baf647a6c3c121a983a59f465da68c5d48ffdc1be907e0d21e906532000000006b483045022100b6aec231b15fb8773238f15c03d77dd96e7ab9effba448379843e6f0cbb26ba80220119e5e30c67b4127889cead6a4670f80f0c1cadbb0c0f190fa97524170e3259c012103e7e1e53c87e4a8c02fb23bd417dfb876d0c3696d152ffb80b1386e248eab4f28ffffffff0165780504000000001976a91450bb2cc36fde08183d0ee60739fa2e6d3104506288ac00000000010000000152fc1244d624b519276d4dd84aea03d6a83c1aa40c851770790cb60e664544af000000006b4830450221008f3ac5dd6c55da5da3eb9cd6bfa528cfa785b280bc5f54a2862fc4e392e1e742022027a68159fe84ac3ce7a17447af01c7d98eb7c60d3ba1bf0d55d5ae9c196c111b01210230d33dbd6caf506e45867d3911e0f3ce8e1ee97f9c29c9061d10c8221b8f1c70ffffffff016c2e9402000000001976a914d833d14b339f88771324bbaf14d6617c1b30733b88ac00000000010000000141952f00a6241223b8a4e98ff265b52765f6d61aa4740139d79ac968b4331a04000000006a47304402206f767726d19315ed548bddfeb21be79ea47692defed45ec9b4d9e89fd11b2e00022077a92029d59f4f4f4346c981a1fc8165cad751206abf351a99f240b791138e030121026fd21992c59314f21e91971840d3f9a420f71ed6c0a809e91b44b96eef2449f8ffffffff0189e24200000000001976a9147bb054ba24861342b791a2ef922438ed450cedf988ac00000000010000000172b6425f1400e2d19ad848be792e5140e5f4d844bc3ef24804590fbf5bd2b71e000000006a4730440220225cc1e41592ed4c59379d1a4cfaa588c37ddc9b2d6a42d73947801ba55707db02203d4533941a6493f8d936f38b47ba1ace239623403386bb943fa8fee0e11d3dc20121025441b1e394761aceac808d6ef76f8b781528da0e80fc5265b9bd3f815b3170d2ffffffff01a66e1908000000001976a914bda2eb7de910bebedbf713a93c7e7231cfa89bf988ac000000000100000001b1a65371429d4206381cbd3d48babecfedad2f690304b0593fab625cf7e5397a000000006b483045022100b07e0b8fcddde4b5ce1ca460d4558fc7565edb5cda912be76210ca72f810cc4502207ad144958b4cb5029fd774f2a3e6b40f52e84d4ccadcf6db1324dadd2487dd8201210231dbf3470c47a21ab267eb43de8592d9c3a3d9f70787daf9ea5b063edd23a96effffffff010c401908000000001976a9142644eb37a330f5f8ab366252d3cc773ae7eadd2b88ac000000000100000001a33316055be60b3226ccd7fc7097ed0043b9087065437c3b6427027b7b1d7532000000006b4830450221008511452392bc4eeee6ff68b7f74ad68f2b6b02073dc8bc408eaa9610e646a4c3022053fa7629e392c13e942ab05f36ee9ec70ad531e46c68605fa3cb8107b6a03d730121026e460dc80ae5c742a10eb13d3bc31fdd790d4e03324acb1f20065d6eb469bc5fffffffff01f8324108000000001976a9145f1cdaf2e9dfb17f268eae4a98fb589bce01f13088ac000000000100000001520b38526a28de3969227d2ca8f1d9324568a275254bbd3b3fa47008cc1588c3000000006b483045022100e75ba26bdb61b35375c487443b6ad81f449901fdebf19256f26b486cf427e49f02206ced0a862e2d0e247d984f497cf89a9d15f6b5dd453f3d198da4c10d29e33a2401210264f9f03caaee83257baaa9d736f07bea09b920725f6d7b60b4a017468d88b586ffffffff015e044108000000001976a914a5410a73769baf960aeebaa14f8aa2559ad975d488ac000000000100000001aeadb39b1ca295329e1f643df5b8de4c906b43564d4797034dd946fe13d5ee18000000006b483045022100cce3db797650e2ede2686f6e0d03b40d050711af00eeebc32ef01717b883774702206e6f2ea81ec3a503ea14d585de377bababd7d3e0dc419a54d6cbae7056c1d551012102a4e3695a49ec679ef92840ee3621587009cad0d931963f1d45d45da8c7c8f9eeffffffff0145712100000000001976a9147bb054ba24861342b791a2ef922438ed450cedf988ac00000000010000000172b6425f1400e2d19ad848be792e5140e5f4d844bc3ef24804590fbf5bd2b71e010000006a47304402201b7a19a9fef7d4493844e7064ac057e381c4478de2117a60dfa9daf4f68cb30a0220632645de5010f81d64349093c71e8de4f46b73066f379abc2cbc4d7d6a91e67d0121028870fe8a4bda06a78bac198ac10403346bca9d97430d32ba2402459bf818dabbffffffff02c6b32f10000000001976a914643e4d6e0bdbc8c93d566e88a61daea595af0c3888acb45b4657000000001976a914c32e5c0cc4218e35e5ff91d9e312e7d585a6878888ac0000000001000000017ebf8b4f47ee55cf0263b91de61b567884248377c45a1f58e7e8ae1dac511bad000000006a47304402203ef11c32daf9c0265c477991df70ba7ac393b404735d653b1a956c2692f1e8c402200dbb35a39a2eb61af7802adb554bad249c90008f3617bc175d4886111e190d94012102b72a5386ff3927f382ca02d5cc5c4d707a49c2c886e9b055ec8bd44e35f117fdffffffff012c852f10000000001976a914a1cef3e6d5049e5d4162e4b003b42d84846f11e688ac00000000010000000175e9fe7d08e2c61f58763cc46508df346ed589c64dfb04ecd030d7181ff91422010000006b483045022100a94a01e85792804266a42446c9c658d3a7566b4326268fa444495473a3cbad02022012cc83c81a8359fb1b3161c2c4ec0586cc847badfd732e9e14c2699dde9f4209012103eef4f80c9aaedbb83c2485b68cdf67fbc85e582d959174116441339a7bcd6c13ffffffff025daf4100000000001976a914a200e7679de27f7711164c57bbfc5d2dd070e6e088acb1e7c300000000001976a914b85e77eda8f84ac82f6af3312d795935495f0e0a88ac000000000100000001aeadb39b1ca295329e1f643df5b8de4c906b43564d4797034dd946fe13d5ee18010000006b4830450221008fdecd50624fb070183a9a26050ff94e1d5894cce6796d032cf6e7576d1d546d0220018c729be78a448c87aed54590be07f4f1f2a884068d199ca6e0dd66047ff2940121032c710b0cfa73d6241614328e21aa579226dfe3817b00499d924d2fe0afcaf087ffffffff02df9f2100000000001976a91494a7ea99c3120813f1f0b5a4a5142b35f42b034288ac6aa26542000000001976a91495ff9353c260c23f32e1bad6e79fc74e419f65e288ac0000000001000000017279ef8e9586a0339dd414df095e102cc8fc7f9b04e40cd59950bf96ce0964a3000000006a4730440220597da6e324579028fffd10cc922eeb91a14ae656ddbfd99c4eaa8c376554964202207e3b9cce552a389cce030233edf271187750fff73d6d4964eb4457e5d09b003a012102a2633f8b8932f63d2a3ca6c8b16be16a82b4f09b888bee695182e913af6412e5ffffffff0145712100000000001976a9147bb054ba24861342b791a2ef922438ed450cedf988ac0000000001000000017279ef8e9586a0339dd414df095e102cc8fc7f9b04e40cd59950bf96ce0964a3010000006a473044022054b192c23dc58baaaeb2b135930fb624adbfd8a157fa036f9f43c4f2543f3e940220166a01282451a82fdd3822e79aa8b03cfadbbdf3871dab9bdb0af91270bf0816012102bbcc6105b9ebd655d7000073d6c759c5c282130d2b18a19c6d4b18dc880f0b63ffffffff023ce71000000000001976a91439a4b2d6988ef503d27487b109842ff7938ca29788ac408b5442000000001976a91449c2ac62967f59194c18757c9e95acebd17977d088ac000000000100000001fff2009607d1f4366eec8551831abfa2e39b0e4f00762a42dd3f297f728ee410000000006b483045022100ecfe045079a2f0ca05260bdfddb17c2b52632d9613d4105c2b1a2d64584883b602201027d3bc2986f6fe67739a45d354f1654bc4372c1bf60392905c11c1b293da4401210290059ee173a66b9d75cb12f3f0e7938175b447dc5b439a43b718f200962981deffffffff01a2b81000000000001976a9147bb054ba24861342b791a2ef922438ed450cedf988ac000000000100000001fff2009607d1f4366eec8551831abfa2e39b0e4f00762a42dd3f297f728ee410010000006b483045022100ddd85228744aa01f79be887bd714dc6612f632449e78bbcfb687d5b75187ec1102206fe52236d7af1221d9301ea2530df74f95849cc47071d31e5a6a306db1e890d30121029f30ec281ae16d358159c575c896dbd011e721c185f92cf04344df50e27e5832ffffffff02080e0200000000001976a9147a2874c4d848d9f42ecfcdd39a23fac58178057d88ac4a4d5242000000001976a914eeb46f3231b4bbb29204bbf241af394412be7cdb88ac000000000100000001247059b560f159554b843d5033bc741041198d064b7eb2b7ac0ef08956cabf26000000006b483045022100ed5a8af307a34ff4ca630a31cb7a734640d3896da3b7e26cd69e480858b5701d02207b801c772f09197a1793ec738fa9aa5abaffdb323e0a53fb1ba49502692259fe012102b1f72e77835a81627963d205e017e77cec61eebe83b9f7cabb59e9c670afa376ffffffff016edf0100000000001976a9147bb054ba24861342b791a2ef922438ed450cedf988ac000000000100000001247059b560f159554b843d5033bc741041198d064b7eb2b7ac0ef08956cabf26010000006b483045022100836aa640017fbfff2e7ecb468f49225c125c3411a269c4b9ded7530b3b20582702204d19d8f0f4da9d0010d2751cbeb7b5e824b5966b7728d37ab48754e96c94a72a012102a34105710a9be266079b86c6568c01da21fbc03ae646b62fe80626ad7af35b96ffffffff02df9f2100000000001976a9140814cfaa1995794437c91f608558ffe29ea11b8488ac7d7d3042000000001976a914cb5e755a8770cc9d744a26860495c37b65cc12c388ac000000000100000001e520c2746d4fe18d9477ed251ff98b12dea6a4d767f01b2796053caf37f92314000000006a47304402205e191ca6f8ed144c5476ea727b3d2fc8751893df21b1917266432b7809fd326d02207990d8f8915adead82dc08458dc808ce2dea6181eca4fe6de8450e287df8f78e0121033510c12001bf6e6d9813aaf66f2671454228070096369c7e2dd55936be47de92ffffffff0145712100000000001976a9147bb054ba24861342b791a2ef922438ed450cedf988ac000000000100000001e520c2746d4fe18d9477ed251ff98b12dea6a4d767f01b2796053caf37f92314010000006a473044022034a4f58599ecafeda6c457eb20b887c00e653e2da20207063bdd91fb9a829b830220293233871d56e1d2afd0737dedf9fe34e9dee61357a1d4c451151aaf7530cab70121021ef6973903dfc4053f8f024ed5d8e6e94f0ba183fdef5b85a36c111744aea2d3ffffffff0223114300000000001976a91471a13c5f320bb1525dba5a03e9bc192bfbd256fd88ac6c3ced41000000001976a914837319ac70eec5a1dc359c428569447f2f1cf88f88ac000000000100000001312e810f0fa893e665aed22c61e0985fa3db7f54ff67c725d1741122c47b83fa000000006b4830450221008d6dffb4ff9bd492008891d762d0eb4ffda931ff4b263d0b0fe1102be12567fb022073605ab1029947c2d3face20813d7a1122d57a9102be66632cc0b0fb30c7cb48012103f9c7e1e008b55116a54f069f78f2d360598bb27554ae473a933c313fb2a83925ffffffff0189e24200000000001976a9147bb054ba24861342b791a2ef922438ed450cedf988ac000000000100000001312e810f0fa893e665aed22c61e0985fa3db7f54ff67c725d1741122c47b83fa010000006a473044022034135c5c7f2a3b7c6272a3dda6d025e23e859b42c28f5fd21123fee5f5897e9c022052bd7583696c5322bd1ff8fbc0b39030764f4d08635bbffef8e7ffc1903480640121020343647edad12814c73c8a045bb4154980d1753532659d94ecd7cb8e38a0cef7ffffffff02df9f2100000000001976a914f466492b2272a44f6b40e73c69bc75914d9a0aef88ac9f6ccb41000000001976a914c2766b9758569e38b7dcc3ebdecec145fe8970b088ac000000000100000001493eca1ff5bf098a2efe1752047b79bbb5e43857a962dab21fe4e96f920a3ce3000000006b483045022100a61c8a7d77b35a3316254c6634fec227b33f5c9f5bb87ea6496abace61fb59a602207ab6e6934c5bca6cc348858a5b7e43304e17778f72d33dddc8bfc70f9011396f012103bb9c968a72da2bf2e747da75eb51ad03a6718e8556ce191234358246c321c9f4ffffffff0145712100000000001976a9147bb054ba24861342b791a2ef922438ed450cedf988ac000000000100000001493eca1ff5bf098a2efe1752047b79bbb5e43857a962dab21fe4e96f920a3ce3010000006b483045022100fccbaa69f580d9624067c10fceee9c0e8c703e7d624ad59a079af07dc12c7db902206fd409564ff0175165e088d73b9f9bee08ed5640f3068978ce353f91b0660411012103350c782464c121c471d003235a065d63c8c3e43c165e66f33ca9ded618d0cf41ffffffff020cb7fe12000000001976a91483696283c09f1f2eaa00ed4ec8f7ea74f2169c7488aca585cc2e000000001976a914c51efea96487ced7029782076b61c3f20029099e88ac00000000010000000160ddcf3d457ff9d76820cf286009101c46d2096dc71ea984fd72e9b2fa43f242000000006b4830450221009c767a536300a8b34e89e8a4a28ca63a06cd1bb7980b3607a2fbf8293c82609a02202f9de00506d979638011badfcf4238e1b5b4bc52c8356fbcac9165f9810ad616012102a85c2f13ccf1d8595e43e01515cf991697fc8fc8a2c62377316b1d8cdc09c55bffffffff017288fe12000000001976a91454ff16ca68f6126ad0800d60a2aae2ba2cf7923288ac000000000100000001ecb0ee801b36a61dfa2558d745c33174012a47d28a628a8b7da1d3b53b7575b6000000006b483045022100ccf2137fcb59c1df4c1c241126a3d0a784bd94fdbac1c259515162875006886b022067bcb9e140228d87c69f989930c7866bb0bacc6f7e8a8e9a5b75c599030c1f7f0121022df22a88dc2c7c7fe9c6495e67e4552e40eb6bb1457a95c52fa9792f6e940a55ffffffff01d859fe12000000001976a9147e91e21b68d5330d77c7a73d97da6300edc8300b88ac000000000100000001e3d0623ca5b3198b25cf17bbe5dbcc21521a346aec06dde555ea13d4039b45f5000000006b483045022100b447e8ade324419aa94acc552026ddd2c5e7a0226b9bfb5f393c9c70a9635f4002203330574079bc7fe298094b2dd430c8fb4b85f32fcfdc9d2679cf62cbb3124d52012102fdb61aebb6e1c63e67ccb1f4cf8817506c18541b3fdb2c3e7213311579cf6b92ffffffff013e2bfe12000000001976a9145f1cdaf2e9dfb17f268eae4a98fb589bce01f13088ac00000000010000000160ddcf3d457ff9d76820cf286009101c46d2096dc71ea984fd72e9b2fa43f242010000006a47304402204c156647df92b3a266c4c1d59fc7abd06fb80e39dee2c163ee8553e6b1843c84022042ed738d9832175cd8de36735e9e391e3b98f67a4c35f6801f2547b93c915b50012103863142a4837cb95bc861bd0dfc32d70427c09cd022cdaebf79ffd91565d60f49ffffffff027db22100000000001976a9149bed6cf5f90dc6842451569fce68fd56bd3c3d5f88ac3aa3aa2e000000001976a91459d7725b4e985d6d051f80981cab9dcc6625121188ac000000000100000001432b4f7f361ab5253abaa7bc5e32159ffa5c550ba97ab1cf36c31a2937cb160e000000006b483045022100aecb5f05dec5dabb59aa52977d74776ac482e4e99dfd2ad25d8c71415818c4c50220541419f6d856dc12f5414223bea14bd3af089727531cc8cba5add7072af5bf82012102335cb1b7caaf4fb2467be3d7c2b02b57b1c6205139dcd7e59da9f1584ab2afb2ffffffff01e3832100000000001976a914980cbc9508e1b8c06f01403179a69478122cd47288ac000000000100000001432b4f7f361ab5253abaa7bc5e32159ffa5c550ba97ab1cf36c31a2937cb160e010000006b483045022100e929409001e22fb4e2f475fff81e39c4ea5e3675e1d144cde703d630f9da284a02205bf5d1e7b611dc99792b12bbb590270af55ba89c2d400d4773b8e5e1765d33f901210336d7edb19a2e1badf221e62c83b65ba66ef6f10c7ea1ea151e77e5607bc397f5ffffffff022df01000000000001976a91452f55ceaec487103fced5dad12772c5c6fa45ad288ac1f83992e000000001976a91493b0d26e27eea6890f234ca184b2752f0fa7b4cc88ac0000000001000000011c7824f76216e2b482469c1aeb39c0b410184709bfc309b77ddb78275da564b4010000006b483045022100fbf1ba79b34250a6d1e36a04ebf041816cde30bb44a4d5510f6207c54fe251c802202fa58fa1ba21886a88d4e9035b932c46ac5897c4d3ead4daf6c3489397db94a001210250982145cfc5830859d81bc964d391c0ee89947e3d77f36cf16a02593286b11bffffffff02c0b12100000000001976a9149157c06dd1da5ee7f2e916ed132a3a6ced91cd9c88ac71a1772e000000001976a914cbdd8fb51247dabfe025ecf26d41844f821a075c88ac0000000001000000011de3b46731fc264adc4a20a7990a4c3c6f63a3ac3b9020dde39bf586385a731d010000006b483045022100b48ba1d921f8f35daa58b8a48a509e47621dc1e867186a20f6aefa3c5423fca102206422fa0b23e24462e352778ebb0fd4dabc544bd9658435b1914cff2fdf9fb79901210265d145c8bd8cc0da64295c12bacf21cbb58d6eddda8e90930fccf8dc8e8de692ffffffff02b513031a000000001976a914f9f5ff36568bddd812d3436d56e1ca20cb3435f088acce5d7414000000001976a9149adfbbd43757c51ce684aa70ab3c72b76c8f84e988ac000000000100000001428cb717681411d3cdf29fba1dc46b39290e6af1e722e9fefb024bdf3e13db07000000006b483045022100d3c0848c63c669b9893ca9b1090b25ef4d9350c10de01571ee91c318727f5a7202202df276043c136c5ed7a91447110d059188c2dc3f556207f19b288340da48d9d7012103cfa06c59c0ed20273b049042f274f49d71d8965511ba1fbc48d15acee964468dffffffff011be5021a000000001976a91416f251f5275547c93990c4cd10002fab0c83133388ac000000000100000001111a4adab899ee324531128132307c70ad4293d3d425b8eaf3a3327e41e05ad7000000006a47304402203b8135852c1a508c3910c8bc2185e175304e854d470139b0ea6f3a2153d3eb9d02200244ffa3036ea3a8fe7706f64922a5a93ee5ae9b1b19e4aa701366365196cf27012103a23ffc6a93d6fb2fc66a6bc2c53574679199f39eca3d0e273a6ef309bd24812fffffffff0181b6021a000000001976a9145d4564393544adff275ae4f18c55f9cce11840bb88ac000000000100000001428cb717681411d3cdf29fba1dc46b39290e6af1e722e9fefb024bdf3e13db07010000006a47304402207180bfcd505e9a89860506ed81963a219c8e7b9dd3de8629eeccdb2ac67cf56f02203a0e428f2a15723e1fe7bcd7da02a4ec48e0314f4e3397b34f378245978b5db0012102db2ff2a4642ce3cd3a1b0ea83dfd2fa631da75d0f53a508b61ccc074a23bacfbffffffff028bf01000000000001976a9145555077735c04b158bd1e6d297ed05a69b227adc88ac553d6314000000001976a9146f4c8d9cfdade3a471e8a683846ee8af754b565e88ac000000000100000001cec2d2de805e6ccff9a3cc4d995dec64e210414084d59d5e028fe50a3b4fc61f010000006b483045022100cafe5e5ba14aadbdadee1f5f49b9f6ed483932ba44a53e1b92c20791c68dd2bb02201ac06b3775128065b2229754e1dc30b2a6c4f92a01d0bcb4623cad200e233918012102b426a6400e323681f3275ce716d296dfd499dda13d8a8c7d5a88ebf0905aff51ffffffff027db22100000000001976a914de6007d7e49bf38a7f27845622064157a503155488acea5a4114000000001976a91479ce541b51809ce53db626bcc0a14e2a8409f9a888ac0000000001000000017ebf8b4f47ee55cf0263b91de61b567884248377c45a1f58e7e8ae1dac511bad010000006b483045022100fcaa51cd52d014a2829c9baa135b5e172ad82c15b647098ad68d9fdee77cf812022047a2bc3346596e6e4f35c8ac0d04bbff29e6eaa72394a759bb96a56e763274e30121035c7a084bb100da2531b7ddd64ad60d0b91e1ab67499e78697e7cab99a0220ffdffffffff02d10ca400000000001976a914e19bc1929d536c157a0f03779d54ddd1221313ce88acf51ea256000000001976a914f816f95a0cfc740811974037818316628b4b365288ac00000000010000000275e9fe7d08e2c61f58763cc46508df346ed589c64dfb04ecd030d7181ff91422000000006b483045022100fb0e2b6931e1b307d88bc5a6337b4062eed6d48565b9b8df0eb4adbcfa1da64302206ae58c61076b0db401a68e4a3c8049318b84839c2036bda5af099873ed6cf11b01210205405bc35b2e079d772c949e3c5fd079aeede2da136632c6ca3b68ebea0f0d44ffffffffe508139da84674426643d068b3559a1668c733b24806ac4ffa94c82983ef2a87000000006b483045022100acf2b3d8a247f571c0408bc9db2a4cbc249c203b97c973079e30918366059ac102203ad9ab646656f6094dd4dea9486b4184c22c925fa5a08116e5607216c741651201210205405bc35b2e079d772c949e3c5fd079aeede2da136632c6ca3b68ebea0f0d44ffffffff015c5d430a000000001976a9147dbcf39ac08a011683cafc13b83c0cbe002ec8bf88ac000000000100000001cd08c3821b624610040b7ad1131069f0481f6dfdb19e8465119e84d6bfc4a23d01000000da0047304402204aabb69f995f6b00592d9bb16d03e3604f9f0dd8e99cbe5884f4e50eb5dc0b4902200acca65b57d9ba058b510c0d7a1a9cfbc1cabe29403cc8e3329512db804d152d0148304502210093a91b44a64531e52d4d7376ea3014595cc6dbed3c3d330e4ec63a26f9d4e79f02203a14cfd3085047b48bc1d3f7b2da808c4b7e3d799d883dd5c9e1edda658e519c0147522102632178d046673c9729d828cfee388e121f497707f810c131e0d3fc0fe0bd66d62103a0951ec7d3a9da9de171617026442fcd30f34d66100fab539853b43f508787d452aeffffffff0240420f000000000017a91478ffd219d40318fd240520f25f01a213f873710287a8e648d20000000017a9148ce5408cfeaddb7ccb2545ded41ef47810945484870000000001000000010baaf1e24daf0cfa9ea9e018f70ffaf1f3b71cd214e85911be275900116aa2af01000000db00483045022100d6523cf1257cca6296e544cffaf7bf2d49cdbc0017076cd8a1d1086b852207d602201eeb183e7497cf18f65fda057f0e2804dfd58ad86d524e937d9a50911a7d05c8014830450221008c7624cc6023d48f922b7f2eb6b95f13935b014573825fea0c80304217614027022028dcbf4deea3f347f6e5d5aefcbcb10f54fba2194bee3d3e5be0cf8ecf7dfe4c0147522102632178d046673c9729d828cfee388e121f497707f810c131e0d3fc0fe0bd66d62103a0951ec7d3a9da9de171617026442fcd30f34d66100fab539853b43f508787d452aeffffffff0240420f000000000017a914a27e666d36b12cb200fcc02b1cd11ef0fd4f626f8770e865d20000000017a9148ce5408cfeaddb7ccb2545ded41ef4781094548487000000000100000004298ca382c13403578afaf1a1b9edd0135ab95c7bd28d0486b230e789912c50a1000000006a47304402207d62df41e680fd24ad768c0433f56911cd46678de9c13a234b1a8cb6cf1ac5d6022002228cd38574df73df8bd8397cc9ecdc398b982ccea39fa3643fd662b213fc7f012102c4d2efb96b99744cdc151f2b17ea39077d96317b79e91340205d6ee5046f08e1ffffffff9445452d4654fd3e773f4ae006d66d516e73cd05bdaaeea8cd1f65a5c0fa4bde000000006b483045022100bac02aafc5c869958fd9104a4b4f6ec06ff2338e07b6175bffdb0f4f2fc8f81d02204414dcd6e63bf79c5af720f7b765efd0352da23edf3854e534f185e13ac0df31012102c4d2efb96b99744cdc151f2b17ea39077d96317b79e91340205d6ee5046f08e1ffffffffc0b8cf0ca63bbebaf405d79caea2f8ec8b42888222b8fa2e80902c9354d184d4000000006b483045022100c78dd55df82812d7c895391367faa0a68fa376142e39d5e8ebe430f496e8082c02200b6a4bfe3b58b2a4e173b7797a446ebeff6e293b23362128ed9ef5842f7c17b4012102c4d2efb96b99744cdc151f2b17ea39077d96317b79e91340205d6ee5046f08e1ffffffffcd1f8d964521586a77226c51bfb38a72d8d97dd65cdb83914403561d7dc6ea74000000006b483045022100cd4d977dd0bb3c1302ea1555495fa4b9667ff48f3ace95e2d6883786606c0e8702203d26c2b15431f41d1c3a3577e9f3ccbddd49c7e14952e7378df555f7beabed95012102c4d2efb96b99744cdc151f2b17ea39077d96317b79e91340205d6ee5046f08e1ffffffff01b0954a0d000000001976a9143ea5c35cd70c66b635f9f3c168b9f8346d86753888ac00000000010000000110244a5d38abe83842fd1ab5d59c53a09d341cd4e06abb4bcbd7cf6a843e346e000000006b483045022100d8060726e7b91cb4495ea5a8c4bf6e30d92d8816c3d7465fdb27ca1f268417f202205f6793c0193ded4113c6488ddfcd34557a632f6b6f9ab1462d8ef04be68b86d9012102ad320a9570f170f43678a73b7fd8eb7430d493b26737461c5f90691060599981ffffffff02f0778e18000000001976a914a520ab4755c0a3b580b180fda094a36315005a1788ac4081ba01000000001976a914f9f51cfecd047e0a8f0b7165b2763f0cc7a66a2688ac0000000001000000020d668d9cee347dd5256beac4d3a6e6cb6aa6ea1af197d0c3ec08359223139e1b00000000171600143f0402fcb8beef28a89be7e94794ec66d469fdb6feffffff0d668d9cee347dd5256beac4d3a6e6cb6aa6ea1af197d0c3ec08359223139e1b010000006b483045022100eaceee00202693d7d54aa84bd8b216f62b1f041b16e08e0352eb1ed50f1ef10102201a90288f667f975cc8d524431d61bd8b15f5d189610d31a9bd12ee37c0674f06012103c4561ce27291b1730e5a429934f45fb4ff9b56e24b6e10e0d176f198fa029671feffffff02506c9800000000001976a9140af575373dad17150d91b6a191a371b59e09d87f88ac001c4e0e0000000017a91403572c975aec5228c0e2982ef8a03b6830e0554f8719510d000100000001dca98db30b62474b7042d2af2d6a35363b87a0cf2eeae4759a180d0e16e89896010000006a473044022066ddee297b922c2975484e62b20fe3550cbfe02ce33944831bc9f374447ab9120220166cda127a4f883518d8d6920bcfa29e1fbaaeb4cc0a0e934b9213fb62e1a68c012102bd4e70324a67d22f98b00affa6a5e3b859525d20e988959bcec1b658e4b25c0efeffffff0280b2e60e000000001976a914e48205d0831462532f9428906e000149ea165b9988acbf2bc142000000001976a9142745c2454bea3ea755efad6c976b23af1f2322d288ac19510d00
17:49:41.543815 DEBUG [blockchain] 0x70000f39d000 block_chain::organize() block hash: 0000000000000521adc862e31310e79d7f98455933833ba2c95d5e4b257a6aef
17:49:41.577694 INFO [node] Block #872730 (00) [0000000000000521adc862e31310e79d7f98455933833ba2c95d5e4b257a6aef] 000.606 17.70% 567540
17:49:41.577989 DEBUG [blockchain] 0x70000f39d000 block_chain::organize() header hash: 000000000000007508f05929b38d07e0f8499d0f17d6a8d709017f5e1218f480
17:49:41.586015 DEBUG [blockchain] 0x70000f39d000 Got next block #872730
17:49:41.597325 DEBUG [blockchain] Verify failed [872730] : invalid witness
 libconsensus : false
 forks        : 16382
 outpoint     : 1b9e1323923508ecc3d097f11aeaa66acbe6a6d3c4ea6b25d57d34ee9c8d660d:0
 script       : a914e5a679f81579cbf20bb2a4764be9e95e8b3e396787
 value        : 100000000
 inpoint      : 118c83b2c7aa4b500e0c0333dbfef990ca6c324004d68f68acb10ddada918474:0
 transaction  : 01000000020d668d9cee347dd5256beac4d3a6e6cb6aa6ea1af197d0c3ec08359223139e1b00000000171600143f0402fcb8beef28a89be7e94794ec66d469fdb6feffffff0d668d9cee347dd5256beac4d3a6e6cb6aa6ea1af197d0c3ec08359223139e1b010000006b483045022100eaceee00202693d7d54aa84bd8b216f62b1f041b16e08e0352eb1ed50f1ef10102201a90288f667f975cc8d524431d61bd8b15f5d189610d31a9bd12ee37c0674f06012103c4561ce27291b1730e5a429934f45fb4ff9b56e24b6e10e0d176f198fa029671feffffff02506c9800000000001976a9140af575373dad17150d91b6a191a371b59e09d87f88ac001c4e0e0000000017a91403572c975aec5228c0e2982ef8a03b6830e0554f8719510d00

JasonCoombs commented 6 years ago

perhaps this is what's missing from protocol_block_sync?

https://github.com/libbitcoin/libbitcoin-node/blob/master/src/protocols/protocol_block_in.cpp#L71-L74

and

https://github.com/libbitcoin/libbitcoin-node/blob/master/src/protocols/protocol_block_in.cpp#L208-L210

I have made these changes and will submit a PR if this solves the problem.

evoskuil commented 6 years ago

The change looks right to me.

JasonCoombs commented 6 years ago

it seems to work. the fix became part of my pending PR here https://github.com/libbitcoin/libbitcoin-node/pull/379 can we get it merged and I'll come back to the libbitcoin-build copyright notice edits?

here are the proposed changes to protocol_block_sync: https://github.com/libbitcoin/libbitcoin-node/pull/379/commits/e6a6896192acc44f774c0f48d25af1c7d233fdf0

evoskuil commented 6 years ago

You should create an independent PR as the copyright commit is large and these are unrelated. Also the PR shouldn't have an internal merge commit ("Merge pull request #1 from libbitcoin/master …"). Create an independent pull request by requesting from a distinct local git branch (or closing the existing PR and reopening later).

JasonCoombs commented 6 years ago

okay

JasonCoombs commented 6 years ago

on closer inspection the witness data is still missing... I'll prepare a PR after I see the witness data. the problem must be easy to fix from here.

JasonCoombs commented 6 years ago

witness data is present during script::verify() but for some reason (still debugging) not yet in storage.

04:33:37.360998 VERBOSE [system] 0x7000083b0000 script::is_pay_to_witness forks: 16382
04:33:37.361025 VERBOSE [system] 0x7000083b0000 script::is_pay_to_witness bien
04:33:37.361051 VERBOSE [system] 0x7000083b0000 script::is_pay_to_witness biwi is false
04:33:37.361077 VERBOSE [system] 0x7000083b0000 script::verify() is_pay_to_script_hash()
04:33:37.361104 VERBOSE [system] 0x7000083b0000 script::verify() embedded_script = zero [3f0402fcb8beef28a89be7e94794ec66d469fdb6]
04:33:37.361164 VERBOSE [system] 0x7000083b0000 script::is_pay_to_witness forks: 16382
04:33:37.361189 VERBOSE [system] 0x7000083b0000 script::is_pay_to_witness bien
04:33:37.361214 VERBOSE [system] 0x7000083b0000 script::is_pay_to_witness biwi
04:33:37.361238 VERBOSE [system] 0x7000083b0000 script::verify() witnessed embedded_script
04:33:37.361262 VERBOSE [system] 0x7000083b0000 script::verify() embedded_script = zero [3f0402fcb8beef28a89be7e94794ec66d469fdb6]
04:33:37.361309 VERBOSE [system] 0x7000083b0000 script::verify() in.script() = [00143f0402fcb8beef28a89be7e94794ec66d469fdb6]
04:33:37.361342 VERBOSE [system] 0x7000083b0000 script::verify() input_witness = [3045022100913d331b78a2c1a2ec0e2af7a826181f26e15e0195cef13f492d2e22b8dd9e2c02201f808621f9684574cbbd755933509f6bce7536aa5c36dd4c57fd635e5485404901] [03e2728bdc007032f5b30c823f4a3cd9236eab5e3a6f23c6ae6be8bd63ff847922]
04:33:37.361399 VERBOSE [system] 0x7000083b0000 witness::extract_embedded_script() called
04:33:37.361436 VERBOSE [system] 0x7000083b0000 stack_ = [3045022100913d331b78a2c1a2ec0e2af7a826181f26e15e0195cef13f492d2e22b8dd9e2c02201f808621f9684574cbbd755933509f6bce7536aa5c36dd4c57fd635e5485404901] [03e2728bdc007032f5b30c823f4a3cd9236eab5e3a6f23c6ae6be8bd63ff847922]
04:33:37.361498 VERBOSE [system] 0x7000083b0000 witness::extract_embedded_script() out_stack.size == 2
04:33:37.361521 VERBOSE [system] 0x7000083b0000 stack_ = [3045022100913d331b78a2c1a2ec0e2af7a826181f26e15e0195cef13f492d2e22b8dd9e2c02201f808621f9684574cbbd755933509f6bce7536aa5c36dd4c57fd635e5485404901] [03e2728bdc007032f5b30c823f4a3cd9236eab5e3a6f23c6ae6be8bd63ff847922]
04:33:37.361598 VERBOSE [system] 0x7000083b0000 witness::verify() called extract_embedded_script() and returned true
04:33:37.361792 VERBOSE [system] 0x7000083b0000 script::verify() finished, returning error::success
04:33:37.361823 VERBOSE [system] 0x7000083b0000 script::verify() for input_index = 1 prevout_script = dup hash160 [e95f772e516a0bb68141e845483e23bc28acd29b] equalverify checksig
04:33:37.361859 VERBOSE [system] 0x7000083b0000 and in.script() = [3045022100eaceee00202693d7d54aa84bd8b216f62b1f041b16e08e0352eb1ed50f1ef10102201a90288f667f975cc8d524431d61bd8b15f5d189610d31a9bd12ee37c0674f0601] [03c4561ce27291b1730e5a429934f45fb4ff9b56e24b6e10e0d176f198fa029671]

JasonCoombs commented 6 years ago

found it. this PR fixes an old bug that prevented witness data from being returned by block.to_data()

https://github.com/libbitcoin/libbitcoin/pull/1050

same bug fix for v3: https://github.com/libbitcoin/libbitcoin/pull/1051

now I see witness data:

09:23:33.527581 DEBUG [node] 0x700009b59000 protocol_block_sync::handle_receive_block() calling reservation_->import() at height: 872730 with block hash = 0000000000000521adc862e31310e79d7f98455933833ba2c95d5e4b257a6aef for block: 00000020a297aeb83865a4d4439b792abedbf153cf3b9d39a17312e0170a000000000000b72796785e6b99195e1717d15eaea41380dd74642a8e0ab828aa8edde65d3f35c99a6d57d8ea0e1a69f0069e2d010000000001010000000000000000000000000000000000000000000000000000000000000000ffffffff2f031a510d0004c99a6d5704b5ce12180cc48f6c57ef1b0f00000000000a636b706f6f6c0b2f416e7453747265616d2fffffffff02672fa812000000001976a914876fbb82ec05caa6af7a3b5e5a983aae6c6cc6d688ac0000000000000000266a24aa21a9edc70ff34ff0984d969c609f88d0dedf2c4489a50d487648ab01aff22bfc20bc750120000000000000000000000000000000000000000000000000000000000000000000000000010000000149f2d3e0baf647a6c3c121a983a59f465da68c5d48ffdc1be907e0d21e906532000000006b483045022100b6aec231b15fb8773238f15c03d77dd96e7ab9effba448379843e6f0cbb26ba80220119e5e30c67b4127889cead6a4670f80f0c1cadbb0c0f190fa97524170e3259c012103e7e1e53c87e4a8c02fb23bd417dfb876d0c3696d152ffb80b1386e248eab4f28ffffffff0165780504000000001976a91450bb2cc36fde08183d0ee60739fa2e6d3104506288ac00000000010000000152fc1244d624b519276d4dd84aea03d6a83c1aa40c851770790cb60e664544af000000006b4830450221008f3ac5dd6c55da5da3eb9cd6bfa528cfa785b280bc5f54a2862fc4e392e1e742022027a68159fe84ac3ce7a17447af01c7d98eb7c60d3ba1bf0d55d5ae9c196c111b01210230d33dbd6caf506e45867d3911e0f3ce8e1ee97f9c29c9061d10c8221b8f1c70ffffffff016c2e9402000000001976a914d833d14b339f88771324bbaf14d6617c1b30733b88ac00000000010000000141952f00a6241223b8a4e98ff265b52765f6d61aa4740139d79ac968b4331a04000000006a47304402206f767726d19315ed548bddfeb21be79ea47692defed45ec9b4d9e89fd11b2e00022077a92029d59f4f4f4346c981a1fc8165cad751206abf351a99f240b791138e030121026fd21992c59314f21e91971840d3f9a420f71ed6c0a809e91b44b96eef2449f8ffffffff0189e24200000000001976a9147bb054ba24861342b791a2ef922438ed450cedf988ac00000000010000000172b6425f1400e2d19ad848be792e5140e5f4d844bc3ef24804590fbf5bd2b71e000000006a4730440220225cc1e41592ed4c59379d1a4cfaa588c37ddc9b2d6a42d73947801ba55707db02203d4533941a6493f8d936f38b47ba1ace239623403386bb943fa8fee0e11d3dc20121025441b1e394761aceac808d6ef76f8b781528da0e80fc5265b9bd3f815b3170d2ffffffff01a66e1908000000001976a914bda2eb7de910bebedbf713a93c7e7231cfa89bf988ac000000000100000001b1a65371429d4206381cbd3d48babecfedad2f690304b0593fab625cf7e5397a000000006b483045022100b07e0b8fcddde4b5ce1ca460d4558fc7565edb5cda912be76210ca72f810cc4502207ad144958b4cb5029fd774f2a3e6b40f52e84d4ccadcf6db1324dadd2487dd8201210231dbf3470c47a21ab267eb43de8592d9c3a3d9f70787daf9ea5b063edd23a96effffffff010c401908000000001976a9142644eb37a330f5f8ab366252d3cc773ae7eadd2b88ac000000000100000001a33316055be60b3226ccd7fc7097ed0043b9087065437c3b6427027b7b1d7532000000006b4830450221008511452392bc4eeee6ff68b7f74ad68f2b6b02073dc8bc408eaa9610e646a4c3022053fa7629e392c13e942ab05f36ee9ec70ad531e46c68605fa3cb8107b6a03d730121026e460dc80ae5c742a10eb13d3bc31fdd790d4e03324acb1f20065d6eb469bc5fffffffff01f8324108000000001976a9145f1cdaf2e9dfb17f268eae4a98fb589bce01f13088ac000000000100000001520b38526a28de3969227d2ca8f1d9324568a275254bbd3b3fa47008cc1588c3000000006b483045022100e75ba26bdb61b35375c487443b6ad81f449901fdebf19256f26b486cf427e49f02206ced0a862e2d0e247d984f497cf89a9d15f6b5dd453f3d198da4c10d29e33a2401210264f9f03caaee83257baaa9d736f07bea09b920725f6d7b60b4a017468d88b586ffffffff015e044108000000001976a914a5410a73769baf960aeebaa14f8aa2559ad975d488ac000000000100000001aeadb39b1ca295329e1f643df5b8de4c906b43564d4797034dd946fe13d5ee18000000006b483045022100cce3db797650e2ede2686f6e0d03b40d050711af00eeebc32ef01717b883774702206e6f2ea81ec3a503ea14d585de377bababd7d3e0dc419a54d6cbae7056c1d551012102a4e3695a49ec679ef92840ee3621587009cad0d931963f1d45d45da8c7c8f9eeffffffff0145712100000000001976a9147bb054ba24861342b791a2ef922438ed450cedf988ac00000000010000000172b6425f1400e2d19ad848be792e5140e5f4d844bc3ef24804590fbf5bd2b71e010000006a47304402201b7a19a9fef7d4493844e7064ac057e381c4478de2117a60dfa9daf4f68cb30a0220632645de5010f81d64349093c71e8de4f46b73066f379abc2cbc4d7d6a91e67d0121028870fe8a4bda06a78bac198ac10403346bca9d97430d32ba2402459bf818dabbffffffff02c6b32f10000000001976a914643e4d6e0bdbc8c93d566e88a61daea595af0c3888acb45b4657000000001976a914c32e5c0cc4218e35e5ff91d9e312e7d585a6878888ac0000000001000000017ebf8b4f47ee55cf0263b91de61b567884248377c45a1f58e7e8ae1dac511bad000000006a47304402203ef11c32daf9c0265c477991df70ba7ac393b404735d653b1a956c2692f1e8c402200dbb35a39a2eb61af7802adb554bad249c90008f3617bc175d4886111e190d94012102b72a5386ff3927f382ca02d5cc5c4d707a49c2c886e9b055ec8bd44e35f117fdffffffff012c852f10000000001976a914a1cef3e6d5049e5d4162e4b003b42d84846f11e688ac00000000010000000175e9fe7d08e2c61f58763cc46508df346ed589c64dfb04ecd030d7181ff91422010000006b483045022100a94a01e85792804266a42446c9c658d3a7566b4326268fa444495473a3cbad02022012cc83c81a8359fb1b3161c2c4ec0586cc847badfd732e9e14c2699dde9f4209012103eef4f80c9aaedbb83c2485b68cdf67fbc85e582d959174116441339a7bcd6c13ffffffff025daf4100000000001976a914a200e7679de27f7711164c57bbfc5d2dd070e6e088acb1e7c300000000001976a914b85e77eda8f84ac82f6af3312d795935495f0e0a88ac000000000100000001aeadb39b1ca295329e1f643df5b8de4c906b43564d4797034dd946fe13d5ee18010000006b4830450221008fdecd50624fb070183a9a26050ff94e1d5894cce6796d032cf6e7576d1d546d0220018c729be78a448c87aed54590be07f4f1f2a884068d199ca6e0dd66047ff2940121032c710b0cfa73d6241614328e21aa579226dfe3817b00499d924d2fe0afcaf087ffffffff02df9f2100000000001976a91494a7ea99c3120813f1f0b5a4a5142b35f42b034288ac6aa26542000000001976a91495ff9353c260c23f32e1bad6e79fc74e419f65e288ac0000000001000000017279ef8e9586a0339dd414df095e102cc8fc7f9b04e40cd59950bf96ce0964a3000000006a4730440220597da6e324579028fffd10cc922eeb91a14ae656ddbfd99c4eaa8c376554964202207e3b9cce552a389cce030233edf271187750fff73d6d4964eb4457e5d09b003a012102a2633f8b8932f63d2a3ca6c8b16be16a82b4f09b888bee695182e913af6412e5ffffffff0145712100000000001976a9147bb054ba24861342b791a2ef922438ed450cedf988ac0000000001000000017279ef8e9586a0339dd414df095e102cc8fc7f9b04e40cd59950bf96ce0964a3010000006a473044022054b192c23dc58baaaeb2b135930fb624adbfd8a157fa036f9f43c4f2543f3e940220166a01282451a82fdd3822e79aa8b03cfadbbdf3871dab9bdb0af91270bf0816012102bbcc6105b9ebd655d7000073d6c759c5c282130d2b18a19c6d4b18dc880f0b63ffffffff023ce71000000000001976a91439a4b2d6988ef503d27487b109842ff7938ca29788ac408b5442000000001976a91449c2ac62967f59194c18757c9e95acebd17977d088ac000000000100000001fff2009607d1f4366eec8551831abfa2e39b0e4f00762a42dd3f297f728ee410000000006b483045022100ecfe045079a2f0ca05260bdfddb17c2b52632d9613d4105c2b1a2d64584883b602201027d3bc2986f6fe67739a45d354f1654bc4372c1bf60392905c11c1b293da4401210290059ee173a66b9d75cb12f3f0e7938175b447dc5b439a43b718f200962981deffffffff01a2b81000000000001976a9147bb054ba24861342b791a2ef922438ed450cedf988ac000000000100000001fff2009607d1f4366eec8551831abfa2e39b0e4f00762a42dd3f297f728ee410010000006b483045022100ddd85228744aa01f79be887bd714dc6612f632449e78bbcfb687d5b75187ec1102206fe52236d7af1221d9301ea2530df74f95849cc47071d31e5a6a306db1e890d30121029f30ec281ae16d358159c575c896dbd011e721c185f92cf04344df50e27e5832ffffffff02080e0200000000001976a9147a2874c4d848d9f42ecfcdd39a23fac58178057d88ac4a4d5242000000001976a914eeb46f3231b4bbb29204bbf241af394412be7cdb88ac000000000100000001247059b560f159554b843d5033bc741041198d064b7eb2b7ac0ef08956cabf26000000006b483045022100ed5a8af307a34ff4ca630a31cb7a734640d3896da3b7e26cd69e480858b5701d02207b801c772f09197a1793ec738fa9aa5abaffdb323e0a53fb1ba49502692259fe012102b1f72e77835a81627963d205e017e77cec61eebe83b9f7cabb59e9c670afa376ffffffff016edf0100000000001976a9147bb054ba24861342b791a2ef922438ed450cedf988ac000000000100000001247059b560f159554b843d5033bc741041198d064b7eb2b7ac0ef08956cabf26010000006b483045022100836aa640017fbfff2e7ecb468f49225c125c3411a269c4b9ded7530b3b20582702204d19d8f0f4da9d0010d2751cbeb7b5e824b5966b7728d37ab48754e96c94a72a012102a34105710a9be266079b86c6568c01da21fbc03ae646b62fe80626ad7af35b96ffffffff02df9f2100000000001976a9140814cfaa1995794437c91f608558ffe29ea11b8488ac7d7d3042000000001976a914cb5e755a8770cc9d744a26860495c37b65cc12c388ac000000000100000001e520c2746d4fe18d9477ed251ff98b12dea6a4d767f01b2796053caf37f92314000000006a47304402205e191ca6f8ed144c5476ea727b3d2fc8751893df21b1917266432b7809fd326d02207990d8f8915adead82dc08458dc808ce2dea6181eca4fe6de8450e287df8f78e0121033510c12001bf6e6d9813aaf66f2671454228070096369c7e2dd55936be47de92ffffffff0145712100000000001976a9147bb054ba24861342b791a2ef922438ed450cedf988ac000000000100000001e520c2746d4fe18d9477ed251ff98b12dea6a4d767f01b2796053caf37f92314010000006a473044022034a4f58599ecafeda6c457eb20b887c00e653e2da20207063bdd91fb9a829b830220293233871d56e1d2afd0737dedf9fe34e9dee61357a1d4c451151aaf7530cab70121021ef6973903dfc4053f8f024ed5d8e6e94f0ba183fdef5b85a36c111744aea2d3ffffffff0223114300000000001976a91471a13c5f320bb1525dba5a03e9bc192bfbd256fd88ac6c3ced41000000001976a914837319ac70eec5a1dc359c428569447f2f1cf88f88ac000000000100000001312e810f0fa893e665aed22c61e0985fa3db7f54ff67c725d1741122c47b83fa000000006b4830450221008d6dffb4ff9bd492008891d762d0eb4ffda931ff4b263d0b0fe1102be12567fb022073605ab1029947c2d3face20813d7a1122d57a9102be66632cc0b0fb30c7cb48012103f9c7e1e008b55116a54f069f78f2d360598bb27554ae473a933c313fb2a83925ffffffff0189e24200000000001976a9147bb054ba24861342b791a2ef922438ed450cedf988ac000000000100000001312e810f0fa893e665aed22c61e0985fa3db7f54ff67c725d1741122c47b83fa010000006a473044022034135c5c7f2a3b7c6272a3dda6d025e23e859b42c28f5fd21123fee5f5897e9c022052bd7583696c5322bd1ff8fbc0b39030764f4d08635bbffef8e7ffc1903480640121020343647edad12814c73c8a045bb4154980d1753532659d94ecd7cb8e38a0cef7ffffffff02df9f2100000000001976a914f466492b2272a44f6b40e73c69bc75914d9a0aef88ac9f6ccb41000000001976a914c2766b9758569e38b7dcc3ebdecec145fe8970b088ac000000000100000001493eca1ff5bf098a2efe1752047b79bbb5e43857a962dab21fe4e96f920a3ce3000000006b483045022100a61c8a7d77b35a3316254c6634fec227b33f5c9f5bb87ea6496abace61fb59a602207ab6e6934c5bca6cc348858a5b7e43304e17778f72d33dddc8bfc70f9011396f012103bb9c968a72da2bf2e747da75eb51ad03a6718e8556ce191234358246c321c9f4ffffffff0145712100000000001976a9147bb054ba24861342b791a2ef922438ed450cedf988ac000000000100000001493eca1ff5bf098a2efe1752047b79bbb5e43857a962dab21fe4e96f920a3ce3010000006b483045022100fccbaa69f580d9624067c10fceee9c0e8c703e7d624ad59a079af07dc12c7db902206fd409564ff0175165e088d73b9f9bee08ed5640f3068978ce353f91b0660411012103350c782464c121c471d003235a065d63c8c3e43c165e66f33ca9ded618d0cf41ffffffff020cb7fe12000000001976a91483696283c09f1f2eaa00ed4ec8f7ea74f2169c7488aca585cc2e000000001976a914c51efea96487ced7029782076b61c3f20029099e88ac00000000010000000160ddcf3d457ff9d76820cf286009101c46d2096dc71ea984fd72e9b2fa43f242000000006b4830450221009c767a536300a8b34e89e8a4a28ca63a06cd1bb7980b3607a2fbf8293c82609a02202f9de00506d979638011badfcf4238e1b5b4bc52c8356fbcac9165f9810ad616012102a85c2f13ccf1d8595e43e01515cf991697fc8fc8a2c62377316b1d8cdc09c55bffffffff017288fe12000000001976a91454ff16ca68f6126ad0800d60a2aae2ba2cf7923288ac000000000100000001ecb0ee801b36a61dfa2558d745c33174012a47d28a628a8b7da1d3b53b7575b6000000006b483045022100ccf2137fcb59c1df4c1c241126a3d0a784bd94fdbac1c259515162875006886b022067bcb9e140228d87c69f989930c7866bb0bacc6f7e8a8e9a5b75c599030c1f7f0121022df22a88dc2c7c7fe9c6495e67e4552e40eb6bb1457a95c52fa9792f6e940a55ffffffff01d859fe12000000001976a9147e91e21b68d5330d77c7a73d97da6300edc8300b88ac000000000100000001e3d0623ca5b3198b25cf17bbe5dbcc21521a346aec06dde555ea13d4039b45f5000000006b483045022100b447e8ade324419aa94acc552026ddd2c5e7a0226b9bfb5f393c9c70a9635f4002203330574079bc7fe298094b2dd430c8fb4b85f32fcfdc9d2679cf62cbb3124d52012102fdb61aebb6e1c63e67ccb1f4cf8817506c18541b3fdb2c3e7213311579cf6b92ffffffff013e2bfe12000000001976a9145f1cdaf2e9dfb17f268eae4a98fb589bce01f13088ac00000000010000000160ddcf3d457ff9d76820cf286009101c46d2096dc71ea984fd72e9b2fa43f242010000006a47304402204c156647df92b3a266c4c1d59fc7abd06fb80e39dee2c163ee8553e6b1843c84022042ed738d9832175cd8de36735e9e391e3b98f67a4c35f6801f2547b93c915b50012103863142a4837cb95bc861bd0dfc32d70427c09cd022cdaebf79ffd91565d60f49ffffffff027db22100000000001976a9149bed6cf5f90dc6842451569fce68fd56bd3c3d5f88ac3aa3aa2e000000001976a91459d7725b4e985d6d051f80981cab9dcc6625121188ac000000000100000001432b4f7f361ab5253abaa7bc5e32159ffa5c550ba97ab1cf36c31a2937cb160e000000006b483045022100aecb5f05dec5dabb59aa52977d74776ac482e4e99dfd2ad25d8c71415818c4c50220541419f6d856dc12f5414223bea14bd3af089727531cc8cba5add7072af5bf82012102335cb1b7caaf4fb2467be3d7c2b02b57b1c6205139dcd7e59da9f1584ab2afb2ffffffff01e3832100000000001976a914980cbc9508e1b8c06f01403179a69478122cd47288ac000000000100000001432b4f7f361ab5253abaa7bc5e32159ffa5c550ba97ab1cf36c31a2937cb160e010000006b483045022100e929409001e22fb4e2f475fff81e39c4ea5e3675e1d144cde703d630f9da284a02205bf5d1e7b611dc99792b12bbb590270af55ba89c2d400d4773b8e5e1765d33f901210336d7edb19a2e1badf221e62c83b65ba66ef6f10c7ea1ea151e77e5607bc397f5ffffffff022df01000000000001976a91452f55ceaec487103fced5dad12772c5c6fa45ad288ac1f83992e000000001976a91493b0d26e27eea6890f234ca184b2752f0fa7b4cc88ac0000000001000000011c7824f76216e2b482469c1aeb39c0b410184709bfc309b77ddb78275da564b4010000006b483045022100fbf1ba79b34250a6d1e36a04ebf041816cde30bb44a4d5510f6207c54fe251c802202fa58fa1ba21886a88d4e9035b932c46ac5897c4d3ead4daf6c3489397db94a001210250982145cfc5830859d81bc964d391c0ee89947e3d77f36cf16a02593286b11bffffffff02c0b12100000000001976a9149157c06dd1da5ee7f2e916ed132a3a6ced91cd9c88ac71a1772e000000001976a914cbdd8fb51247dabfe025ecf26d41844f821a075c88ac0000000001000000011de3b46731fc264adc4a20a7990a4c3c6f63a3ac3b9020dde39bf586385a731d010000006b483045022100b48ba1d921f8f35daa58b8a48a509e47621dc1e867186a20f6aefa3c5423fca102206422fa0b23e24462e352778ebb0fd4dabc544bd9658435b1914cff2fdf9fb79901210265d145c8bd8cc0da64295c12bacf21cbb58d6eddda8e90930fccf8dc8e8de692ffffffff02b513031a000000001976a914f9f5ff36568bddd812d3436d56e1ca20cb3435f088acce5d7414000000001976a9149adfbbd43757c51ce684aa70ab3c72b76c8f84e988ac000000000100000001428cb717681411d3cdf29fba1dc46b39290e6af1e722e9fefb024bdf3e13db07000000006b483045022100d3c0848c63c669b9893ca9b1090b25ef4d9350c10de01571ee91c318727f5a7202202df276043c136c5ed7a91447110d059188c2dc3f556207f19b288340da48d9d7012103cfa06c59c0ed20273b049042f274f49d71d8965511ba1fbc48d15acee964468dffffffff011be5021a000000001976a91416f251f5275547c93990c4cd10002fab0c83133388ac000000000100000001111a4adab899ee324531128132307c70ad4293d3d425b8eaf3a3327e41e05ad7000000006a47304402203b8135852c1a508c3910c8bc2185e175304e854d470139b0ea6f3a2153d3eb9d02200244ffa3036ea3a8fe7706f64922a5a93ee5ae9b1b19e4aa701366365196cf27012103a23ffc6a93d6fb2fc66a6bc2c53574679199f39eca3d0e273a6ef309bd24812fffffffff0181b6021a000000001976a9145d4564393544adff275ae4f18c55f9cce11840bb88ac000000000100000001428cb717681411d3cdf29fba1dc46b39290e6af1e722e9fefb024bdf3e13db07010000006a47304402207180bfcd505e9a89860506ed81963a219c8e7b9dd3de8629eeccdb2ac67cf56f02203a0e428f2a15723e1fe7bcd7da02a4ec48e0314f4e3397b34f378245978b5db0012102db2ff2a4642ce3cd3a1b0ea83dfd2fa631da75d0f53a508b61ccc074a23bacfbffffffff028bf01000000000001976a9145555077735c04b158bd1e6d297ed05a69b227adc88ac553d6314000000001976a9146f4c8d9cfdade3a471e8a683846ee8af754b565e88ac000000000100000001cec2d2de805e6ccff9a3cc4d995dec64e210414084d59d5e028fe50a3b4fc61f010000006b483045022100cafe5e5ba14aadbdadee1f5f49b9f6ed483932ba44a53e1b92c20791c68dd2bb02201ac06b3775128065b2229754e1dc30b2a6c4f92a01d0bcb4623cad200e233918012102b426a6400e323681f3275ce716d296dfd499dda13d8a8c7d5a88ebf0905aff51ffffffff027db22100000000001976a914de6007d7e49bf38a7f27845622064157a503155488acea5a4114000000001976a91479ce541b51809ce53db626bcc0a14e2a8409f9a888ac0000000001000000017ebf8b4f47ee55cf0263b91de61b567884248377c45a1f58e7e8ae1dac511bad010000006b483045022100fcaa51cd52d014a2829c9baa135b5e172ad82c15b647098ad68d9fdee77cf812022047a2bc3346596e6e4f35c8ac0d04bbff29e6eaa72394a759bb96a56e763274e30121035c7a084bb100da2531b7ddd64ad60d0b91e1ab67499e78697e7cab99a0220ffdffffffff02d10ca400000000001976a914e19bc1929d536c157a0f03779d54ddd1221313ce88acf51ea256000000001976a914f816f95a0cfc740811974037818316628b4b365288ac00000000010000000275e9fe7d08e2c61f58763cc46508df346ed589c64dfb04ecd030d7181ff91422000000006b483045022100fb0e2b6931e1b307d88bc5a6337b4062eed6d48565b9b8df0eb4adbcfa1da64302206ae58c61076b0db401a68e4a3c8049318b84839c2036bda5af099873ed6cf11b01210205405bc35b2e079d772c949e3c5fd079aeede2da136632c6ca3b68ebea0f0d44ffffffffe508139da84674426643d068b3559a1668c733b24806ac4ffa94c82983ef2a87000000006b483045022100acf2b3d8a247f571c0408bc9db2a4cbc249c203b97c973079e30918366059ac102203ad9ab646656f6094dd4dea9486b4184c22c925fa5a08116e5607216c741651201210205405bc35b2e079d772c949e3c5fd079aeede2da136632c6ca3b68ebea0f0d44ffffffff015c5d430a000000001976a9147dbcf39ac08a011683cafc13b83c0cbe002ec8bf88ac000000000100000001cd08c3821b624610040b7ad1131069f0481f6dfdb19e8465119e84d6bfc4a23d01000000da0047304402204aabb69f995f6b00592d9bb16d03e3604f9f0dd8e99cbe5884f4e50eb5dc0b4902200acca65b57d9ba058b510c0d7a1a9cfbc1cabe29403cc8e3329512db804d152d0148304502210093a91b44a64531e52d4d7376ea3014595cc6dbed3c3d330e4ec63a26f9d4e79f02203a14cfd3085047b48bc1d3f7b2da808c4b7e3d799d883dd5c9e1edda658e519c0147522102632178d046673c9729d828cfee388e121f497707f810c131e0d3fc0fe0bd66d62103a0951ec7d3a9da9de171617026442fcd30f34d66100fab539853b43f508787d452aeffffffff0240420f000000000017a91478ffd219d40318fd240520f25f01a213f873710287a8e648d20000000017a9148ce5408cfeaddb7ccb2545ded41ef47810945484870000000001000000010baaf1e24daf0cfa9ea9e018f70ffaf1f3b71cd214e85911be275900116aa2af01000000db00483045022100d6523cf1257cca6296e544cffaf7bf2d49cdbc0017076cd8a1d1086b852207d602201eeb183e7497cf18f65fda057f0e2804dfd58ad86d524e937d9a50911a7d05c8014830450221008c7624cc6023d48f922b7f2eb6b95f13935b014573825fea0c80304217614027022028dcbf4deea3f347f6e5d5aefcbcb10f54fba2194bee3d3e5be0cf8ecf7dfe4c0147522102632178d046673c9729d828cfee388e121f497707f810c131e0d3fc0fe0bd66d62103a0951ec7d3a9da9de171617026442fcd30f34d66100fab539853b43f508787d452aeffffffff0240420f000000000017a914a27e666d36b12cb200fcc02b1cd11ef0fd4f626f8770e865d20000000017a9148ce5408cfeaddb7ccb2545ded41ef4781094548487000000000100000004298ca382c13403578afaf1a1b9edd0135ab95c7bd28d0486b230e789912c50a1000000006a47304402207d62df41e680fd24ad768c0433f56911cd46678de9c13a234b1a8cb6cf1ac5d6022002228cd38574df73df8bd8397cc9ecdc398b982ccea39fa3643fd662b213fc7f012102c4d2efb96b99744cdc151f2b17ea39077d96317b79e91340205d6ee5046f08e1ffffffff9445452d4654fd3e773f4ae006d66d516e73cd05bdaaeea8cd1f65a5c0fa4bde000000006b483045022100bac02aafc5c869958fd9104a4b4f6ec06ff2338e07b6175bffdb0f4f2fc8f81d02204414dcd6e63bf79c5af720f7b765efd0352da23edf3854e534f185e13ac0df31012102c4d2efb96b99744cdc151f2b17ea39077d96317b79e91340205d6ee5046f08e1ffffffffc0b8cf0ca63bbebaf405d79caea2f8ec8b42888222b8fa2e80902c9354d184d4000000006b483045022100c78dd55df82812d7c895391367faa0a68fa376142e39d5e8ebe430f496e8082c02200b6a4bfe3b58b2a4e173b7797a446ebeff6e293b23362128ed9ef5842f7c17b4012102c4d2efb96b99744cdc151f2b17ea39077d96317b79e91340205d6ee5046f08e1ffffffffcd1f8d964521586a77226c51bfb38a72d8d97dd65cdb83914403561d7dc6ea74000000006b483045022100cd4d977dd0bb3c1302ea1555495fa4b9667ff48f3ace95e2d6883786606c0e8702203d26c2b15431f41d1c3a3577e9f3ccbddd49c7e14952e7378df555f7beabed95012102c4d2efb96b99744cdc151f2b17ea39077d96317b79e91340205d6ee5046f08e1ffffffff01b0954a0d000000001976a9143ea5c35cd70c66b635f9f3c168b9f8346d86753888ac00000000010000000110244a5d38abe83842fd1ab5d59c53a09d341cd4e06abb4bcbd7cf6a843e346e000000006b483045022100d8060726e7b91cb4495ea5a8c4bf6e30d92d8816c3d7465fdb27ca1f268417f202205f6793c0193ded4113c6488ddfcd34557a632f6b6f9ab1462d8ef04be68b86d9012102ad320a9570f170f43678a73b7fd8eb7430d493b26737461c5f90691060599981ffffffff02f0778e18000000001976a914a520ab4755c0a3b580b180fda094a36315005a1788ac4081ba01000000001976a914f9f51cfecd047e0a8f0b7165b2763f0cc7a66a2688ac00000000010000000001020d668d9cee347dd5256beac4d3a6e6cb6aa6ea1af197d0c3ec08359223139e1b00000000171600143f0402fcb8beef28a89be7e94794ec66d469fdb6feffffff0d668d9cee347dd5256beac4d3a6e6cb6aa6ea1af197d0c3ec08359223139e1b010000006b483045022100eaceee00202693d7d54aa84bd8b216f62b1f041b16e08e0352eb1ed50f1ef10102201a90288f667f975cc8d524431d61bd8b15f5d189610d31a9bd12ee37c0674f06012103c4561ce27291b1730e5a429934f45fb4ff9b56e24b6e10e0d176f198fa029671feffffff02506c9800000000001976a9140af575373dad17150d91b6a191a371b59e09d87f88ac001c4e0e0000000017a91403572c975aec5228c0e2982ef8a03b6830e0554f8702483045022100913d331b78a2c1a2ec0e2af7a826181f26e15e0195cef13f492d2e22b8dd9e2c02201f808621f9684574cbbd755933509f6bce7536aa5c36dd4c57fd635e54854049012103e2728bdc007032f5b30c823f4a3cd9236eab5e3a6f23c6ae6be8bd63ff8479220019510d000100000001dca98db30b62474b7042d2af2d6a35363b87a0cf2eeae4759a180d0e16e89896010000006a473044022066ddee297b922c2975484e62b20fe3550cbfe02ce33944831bc9f374447ab9120220166cda127a4f883518d8d6920bcfa29e1fbaaeb4cc0a0e934b9213fb62e1a68c012102bd4e70324a67d22f98b00affa6a5e3b859525d20e988959bcec1b658e4b25c0efeffffff0280b2e60e000000001976a914e48205d0831462532f9428906e000149ea165b9988acbf2bc142000000001976a9142745c2454bea3ea755efad6c976b23af1f2322d288ac19510d00

JasonCoombs commented 6 years ago

with these fixes made it doesn't appear to be organizing blocks (validating transactions) above the highest checkpoint. not sure why not. still debugging this issue.

evoskuil commented 6 years ago

Is this issue resolved? If so please close, and open another for other issues as necessary.

JasonCoombs commented 6 years ago

I'm not 100% certain yet that we've resolved the bip141 problem. There's witness data now, as in v3, but I haven't seen transaction validation occur above the highest checkpoint block. This might be a separate issue, as you suggest, but it could be that my fix is not stopping the channel correctly when a peer is not configured for segwit support? after more than two dozen tries since yesterday I'm almost sync'ed current on Testnet, but this is by using a new checkpoint hash near to the latest block not a result of seeing full bip141 functionality during validation (Block Organization).

A variety of failure conditions occur, requiring delete of the blockchain directory and copy from backup prior to the failure, in order to get to this point; the difficulty keeping the synchronization process going is a separate issue, but this error (see below) could be related to the way in which non-segwit peers are being rejected by my proposed bug fix, which was a step and line of code that I simply guessed was correct (stopping the channel) in the case where local segwit activation is detected and remote segwit activation is expected.

20:11:22.953827 INFO [node] Block #1380650 (00) [00000000000002a38397eade89d8b99fa1fe70a9e12ed790ddc6e06120538ddd] 000.956 01.17% 33783
20:11:22.984759 INFO [blockchain] 0x7000081db000 Organized blocks [1380650-1380650]
20:11:23.707276 INFO [blockchain] 0x7000081db000 Organized blocks [1380651-1380651]
20:11:24.167350 INFO [blockchain] 0x7000081db000 Organized blocks [1380652-1380652]
20:11:32.218064 INFO [blockchain] 0x7000081db000 Organized blocks [1380653-1380653]
20:11:49.882060 INFO [network] Connected outbound channel [18.136.76.196:18333] (1)
20:12:39.632403 INFO [network] Connected outbound channel [58.229.208.152:18333] (1)
20:13:27.646916 INFO [network] Connected outbound channel [13.124.0.147:18333] (1)
Segmentation fault: 11

evoskuil commented 6 years ago

I’ve validated about half of testnet without checkpoints. The only problem I’ve seen is a consequence of reorgs.

JasonCoombs commented 6 years ago

segwit doesn't start until block 872730 and I agree that block organization is working well up to that point, other than excessive duplicates and the occasional peer that supplies non-BTC blocks (when encountered I am just adding them to my blacklist).

libbitcoin / libbitcoin-node

[master] Testnet activation of bip141 failing? #383