Howto?

[Sjors]

Can you add some documentation on how to use this?

I'd like to build the full index on mainnet. For now without cut through, because I want to compare as many potential edge cases as possible with the result of https://github.com/bitcoin/bitcoin/pull/28241).

Also ideally without Docker.

[setavenger]

Do you want to make changes? The full index is produced as well, just not sorted. Currently both a cut-through and standard index are being produced, both are listed here.

Nonetheless I will write a how-to and also make the configuration easier without having to change code or similar hurdles.

[Sjors]

It only goes to block 834,761 though.

[setavenger]

Fair enough, will do the How-To today. I had already started on streamlining the config anyways.

[setavenger]

@Sjors Added build and run instructions. You can also set a SYNC_START_HEIGHT to avoid unnecessarily looping through pre taproot blocks.

I'll keep this issue open for now in case some details are missing.

[Sjors]

Building the index now...

Mmm, it fails to build:

2024/04/22 17:12:25.529072 routine.go:124: [INFO] successfully processed block: 712516
2024/04/22 17:12:25.529153 routine.go:91: [INFO] Processing block: 712517
2024/04/22 17:12:25.531737 routine.go:124: [INFO] successfully processed block: 712517
2024/04/22 17:12:25.531806 routine.go:91: [INFO] Processing block: 712518
2024/04/22 17:12:25.541117 /home/sjors/dev/BlindBit-Backend/src/db/dblevel/tweak.go:46: [ERROR] []
2024/04/22 17:12:25.541317 /home/sjors/dev/BlindBit-Backend/src/db/dblevel/tweak.go:47: [ERROR] number of tweaks was not exactly 1
2024/04/22 17:12:25.541359 /home/sjors/dev/BlindBit-Backend/src/core/cleanup.go:81: [ERROR] number of tweaks was not exactly 1
2024/04/22 17:12:25.541403 /home/sjors/dev/BlindBit-Backend/src/core/routine.go:169: [ERROR] number of tweaks was not exactly 1
2024/04/22 17:12:25.541454 /home/sjors/dev/BlindBit-Backend/src/core/routine.go:98: [ERROR] number of tweaks was not exactly 1

Also, once I do manage to build it, how do I go about getting tweaks for a given block?

I guess what I'd want to do it have a script that iterates over all blocks and then checks your tweaks against the Bitcoin Core RPC call results, maybe with some jq magic.

[setavenger]

Mmm, it fails to build:

This looks like one of the newer features I implemented. I can probably exclude it for now as it's not relevant for what you are trying to do. Then you won't have to wait for me to fix the underlying cause.

Also, once I do manage to build it, how do I go about getting tweaks for a given block?

There is an API exposed once the index server is fully synced up. I'll add the specification to the README. It's simply something like this localhost:8000/tweaks/840000 and returns an array of tweaks. Default is port 8000, something that should be made variable soon as well.

[setavenger]

@Sjors you can now use TWEAKS_ONLY=1 to compute tweaks only. This also circumvents the error.

[Sjors]

Much better, but it crashes at 838,781:

2024/04/23 17:17:34.847956 routine.go:91: [INFO] Processing block: 838780
2024/04/23 17:17:34.886034 routine.go:124: [INFO] successfully processed block: 838780
2024/04/23 17:17:34.886063 routine.go:91: [INFO] Processing block: 838781
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x58 pc=0x731640]

goroutine 290083184 [running]:
SilentPaymentAppBackend/src/core.extractPubKeys({{0xc004434080, 0x40}, {0xc0044340c0, 0x40}, 0x2, {0xc033542230, 0x1, 0x1}, {0xc024b04480, 0x3, ...}})
    /home/sjors/dev/BlindBit-Backend/src/core/tweak.go:311 +0xa0
SilentPaymentAppBackend/src/core.ComputeTweakPerTx({{0xc004434080, 0x40}, {0xc0044340c0, 0x40}, 0x2, {0xc033542230, 0x1, 0x1}, {0xc024b04480, 0x3, ...}})
    /home/sjors/dev/BlindBit-Backend/src/core/tweak.go:265 +0x58
SilentPaymentAppBackend/src/core.ComputeTweaksForBlockV2.func1({{0xc004434080, 0x40}, {0xc0044340c0, 0x40}, 0x2, {0xc033542230, 0x1, 0x1}, {0xc024b04480, 0x3, ...}})
    /home/sjors/dev/BlindBit-Backend/src/core/tweak.go:197 +0x1b9
created by SilentPaymentAppBackend/src/core.ComputeTweaksForBlockV2 in goroutine 77
    /home/sjors/dev/BlindBit-Backend/src/core/tweak.go:187 +0xf8

[setavenger]

@Sjors fixed. Not quite sure how this error popped up this late. It seems as if this was the first block with a taproot output. I hadn't synced this far yet.

[setavenger]

Added a tweak-index endpoint, the existing endpoint always used the cut-through data.

[Sjors]

Always happy to break things :-) Running it again now, from start.

[setavenger]

Much appreciated! I have been the only one running this until now. Any bug report and feedback is very welcome :)

[Sjors]

Works! I'll open new issues for specific problems, if any.