v27 node thinks my v28 / master block database is corrupted

Sjors commented 2 weeks ago

Is there an existing issue for this?

[X] I have searched the existing issues

Current behaviour

I'm unable to start Bitcoin Core v27.1 or v27.2 (haven't tried older) on my newly installed node. The node works with v28.0 and on recent master (2b33322169bc).

I never set -blocksxor.

I did use assume utxo during the initial sync. I haven't tried (yet) to sync again and see if that matters.

The initial sync was done on master, I haven't tried doing the sync with the release instead.

Expected behaviour

Older node software should work, unless -blocksxor was used.

Steps to reproduce

Start Bitcoin Core v28.0 release binary. Stop and wait. Start v27.2 release binary. It asks if I want to rebuild the blocks database. Dismiss. Start v28.0 again and it's fine.

Relevant log output

2024-11-13T20:28:42Z Assuming ancestors of block 000000000000000000026811d149d4d261995ec5b3f64f439a0a10e1a464af9a have valid signatures.
2024-11-13T20:28:42Z Setting nMinimumChainWork=000000000000000000000000000000000000000063c4ebd298db40af57541800
2024-11-13T20:28:42Z Opening LevelDB in /Users/sjors/Library/Application Support/Bitcoin/blocks/index
2024-11-13T20:28:42Z Opened LevelDB successfully
2024-11-13T20:28:42Z Using obfuscation key for /Users/sjors/Library/Application Support/Bitcoin/blocks/index: 0000000000000000
2024-11-13T20:28:43Z LoadBlockIndexDB: last block file = 4608
2024-11-13T20:28:43Z LoadBlockIndexDB: last block file info: CBlockFileInfo(blocks=21, size=32956714, heights=870155...870175, time=2024-11-13...2024-11-13)
2024-11-13T20:28:43Z Checking all blk files are present...
2024-11-13T20:28:44Z Initializing chainstate Chainstate [ibd] @ height -1 (null)
2024-11-13T20:28:44Z Opening LevelDB in /Users/sjors/Library/Application Support/Bitcoin/chainstate
2024-11-13T20:28:44Z Opened LevelDB successfully
2024-11-13T20:28:44Z Using obfuscation key for /Users/sjors/Library/Application Support/Bitcoin/chainstate: 59783779d122abf8
2024-11-13T20:28:44Z Loaded best chain: hashBestChain=00000000000000000000bc7bf3111f6d21d7bb69f2d4f5f0132803adb42e5314 height=870175 date=2024-11-13T20:14:08Z progress=0.999995
2024-11-13T20:28:44Z Opening LevelDB in /Users/sjors/Library/Application Support/Bitcoin/chainstate
2024-11-13T20:28:44Z Opened LevelDB successfully
2024-11-13T20:28:44Z Using obfuscation key for /Users/sjors/Library/Application Support/Bitcoin/chainstate: 59783779d122abf8
2024-11-13T20:28:44Z [Chainstate [ibd] @ height 870175 (00000000000000000000bc7bf3111f6d21d7bb69f2d4f5f0132803adb42e5314)] resized coinsdb cache to 8.0 MiB
2024-11-13T20:28:44Z [Chainstate [ibd] @ height 870175 (00000000000000000000bc7bf3111f6d21d7bb69f2d4f5f0132803adb42e5314)] resized coinstip cache to 16374.0 MiB
2024-11-13T20:28:44Z init message: Blokken controleren...
2024-11-13T20:28:44Z Verifying last 6 blocks at level 3
2024-11-13T20:28:44Z Verification progress: 0%
2024-11-13T20:28:44Z ERROR: ReadBlockFromDisk: Deserialize or I/O error - ReadCompactSize(): size too large: unspecified iostream_category error at FlatFilePos(nFile=4608, nPos=31381085)
2024-11-13T20:28:44Z Verification error: ReadBlockFromDisk failed at 870175, hash=00000000000000000000bc7bf3111f6d21d7bb69f2d4f5f0132803adb42e5314
2024-11-13T20:28:44Z : Corrupted block database detected.
Please restart with -reindex or -reindex-chainstate to recover.
2024-11-13T20:29:04Z Aborted block database rebuild. Exiting.

How did you obtain Bitcoin Core

Compiled from source

What version of Bitcoin Core are you using?

master@2b33322169bc, v28.0, v27.2

Operating system and version

macOS 15.1

Machine specifications

Unpruned, everything stored on the built-in SSD.

Sjors commented 2 weeks ago

Although I did not use -blocksxor there is a xor.dat file in my blocks dir.

Here's the block, undo and xor file: https://download.sprovoost.nl/download.php?id=10&token=34c0e1dcec8e6d096c2f25ebfe8f59f2

maflcko commented 2 weeks ago

Although I did not use -blocksxor there is a xor.dat file in my blocks dir.

This is expected. You can print the help (and the default values) of args with the -help arg.

maflcko commented 2 weeks ago

I can see the point of delaying the default value by one release, but creating a blocksdir with the most recent release, only to downgrade seems an edge case.

Sjors commented 2 weeks ago

Too late now, but I think the release notes should have made it more clear that you can't downgrade.

https://bitcoincore.org/en/releases/28.0/

Burried under "Low-level Changes":

Blockstorage

Block files are now XOR’d by default with a key stored in the blocksdir. Previous releases of Bitcoin Core or previous external software will not be able to read the blocksdir with a non-zero XOR-key.

Sjors commented 2 weeks ago

Oh never mind, that's only "for a freshly initialized blocksdir". That seems reasonable.

bitcoin / bitcoin