Start with a pre-synced chain

[willcl-ark]

with $$$

[josibake]

I think we will need this (or something like it) for Tabconf, especially if transaction creation is apart of any of the attacks. Adding the tabconf milestone.

[pinheadmz]

Even if theres only one [signet] miner, tx_flood will still spread out the coins over time. We can also do that manually. For Tabconf I think we should have the cluster humming along several hours before the event starts. We can practice a few deployments over the next few weeks. I think a presynced chain makes more sense for rapid-fire time-sensitive operations (like hundreds of functional tests) but is less critical for Warnet since it really only adds a few minutes in the worst case.

[josibake]

tx_flood will still spread out the coins over time

I don't think this is true with how tx_flood is currently written. It looks like tx flood assumes the node is itself a miner, generates some random addresses for the node itself and then sends to itself.

We can also do that manually

Also fine with a manual script for pre-generating coins just for Tabconf if a pre-synced chain will take too long to get working. In general, I'm trying to find two birds one stone opportunities where if something is a) generally good for warnet and b) solves a problem for Tabconf, I'd prefer that over a custom script only useful for Tabconf. But if pre-synced chains is not the right approach or will take too long, then I can open an issue for manually populating tanks for warnet.

edit: ref for tx_flood: https://github.com/bitcoin-dev-project/warnet/blob/20da5b4848def0f7e50786941c0797bc4f1d9e77/resources/scenarios/tx_flood.py#L32-L50

[willcl-ark]

It spreads out the coins output-wise, but you are correct it's a self-send.

IIRC correctly (my mind is hazy), I checked out the regtest code from a pre-mined chain and it's ... actually just the first node doing a quick pre-mine of it's own, then adding the privkeys to nodes 0-3. No actual, pre-mined chain (as it fixed datadir) is going on. Will try and find the code again in a bit to double-check

[willcl-ark]

Here it is. I do like how they mine first blocks in the past to get out of IBD quicker (we should do that!)

https://github.com/bitcoin/bitcoin/blob/0533e652355a68a2c52fb1524a8033bbb5bfa837/test/functional/test_framework/test_framework.py#L823-L895

[josibake]

Nice! Seems like we could port the _initialize_chain code, where a single node (controlled by us, e.g., the signet miner) starts with a pre-initialised chain, distributes the coins between N nodes and spits out M private keys. Those private keys can then be given to each of the tanks. This way, tanks come online, do IBD up to 199 blocks, and already have private keys loaded into their wallets (passed as a config argument). This would be slightly manual in that we need to run the initialisation step first and then pass the private key arguments to each of the tanks, but feels like we could easily automate in the future.

[pinheadmz]

I don't think this is true with how tx_flood is currently written. It looks like tx flood assumes the node is itself a miner, generates some random addresses for the node itself and then sends to itself.

It spreads out the coins output-wise, but you are correct it's a self-send.

No, the logic is:

1. create a big empty list of addresses
2. for each node, create wallet (just so happens to be called "miner") generate addresses and add them to the list
3. in one thread per node, in an endless loop, check the node wallet balance and if there is any money there, send a tx to random addresses from that big list

SO this can start at any point and will only send txs from nodes that have a wallet balance. even if there is only one miner, the money will eventually be spread out to everyone

[willcl-ark]

So I think we can close this?

[josibake]

SO this can start at any point and will only send txs from nodes that have a wallet balance. even if there is only one miner, the money will eventually be spread out to everyone

I see, thanks for the explanation. I didn't look at the code for ensure_miner and assumed it was doing something more sophisticated. Also wasn't apparent to me that new addresses keep filling up self.addrs from all nodes.

TBH, still not convinced this is a good idea for seeding the network since I don't have an intuition on how long this takes to actually ensure every node has a sufficient amount of coins, and also seems hard to guarantee everyone has a similar sized amount. I'd be worried about a scenario where Team A gets way more utxos than Team B , so then Team B needs a top up at some point while team A gets to keep attacking. I also think we should avoid "we'll just manually do X" as much as possible, because this will make actually running warnet on the day of a nightmare. I also think we should be prepared for a catastrophic failure where we need to restart the cluster. If the only way to get everyone UTXOS is us manually doing a bunch the day transactions or waiting for tx_flood to fairly distribute coins, a restart is not gonna be pleasant.

I'd recommend we pursue the _initialize_chain route and perhaps time box it to a few days. If it doesn't work, we can use the tx_flood method or write a manual script. Happy to take a crack at this myself.

[willcl-ark]

What would be the difference between that and what we have?

[josibake]

What would be the difference between that and what we have?

https://github.com/bitcoin-dev-project/warnet/issues/466#issuecomment-2324802706

[willcl-ark]

What would be the difference between that and what we have?

#466 (comment)

So currently with miner_std:

• Start all nodes
• Mine mature blocks on all of them
• They both recieve coins and sync at the same time

In #466 (comment):

• Mine first
• Sync next
• Hand out keys last

What's the material difference I'm missing?

[josibake]

Mine mature blocks on all of them

this doesn't work for Tabconf if we a) limit access to generate commands for players or b) have a working signet. The whole idea of a presynced chain is that the UTXO set can be deterministically handed out to players without requiring players to be miners.

[willcl-ark]

Mine mature blocks on all of them

this doesn't work for Tabconf if we a) limit access to generate commands for players or b) have a working signet. The whole idea of a presynced chain is that the UTXO set can be deterministically handed out to players without requiring players to be miners.

My question was about how -- in the absence of a backported signet, or limiting generate access -- the premined chain solution differs from the current solution in miner_std.

Naturally if we have either of those solutions then this whole discussion is moot, and we still don't need a premined chain, as we can just distribute funds/blocks at will.

[josibake]

Right, what's missing from this issue is some pre-conversation in the warnet chat where we were discussing whether or not to just use tx_flood/mining_std for distributing coins for Tabconf. As I've laid out here https://github.com/bitcoin-dev-project/warnet/issues/466#issuecomment-2326913937, I don't think either of those are viable options:

• mining_std requires everyone to be a miner
• tx_flood would take an indeterminate amount of time to distribute coins and no guarantees around distributing coins evenly

I think we should build towards having a signet miner, and in that case yes, we can write a manual script just for Tabconf for manually distributing coins. Personally, I still see value in have a pre-mined chain with coins pre-allocated as this allows for better reproducibility. If your argument is "we don't need this for Tabconf," great, let's remove the milestone and leave the issue open. If your argument is "we don't need pre-mined chains for warnet at all," I disagree.

[pinheadmz]

I'm going to run a 100-tank warnet in GKE today with one miner and tx_flood and monitor block count and wallet balance over time, that should give us at least a little something to go on.

[pinheadmz]

After 5 minutes, all 100 tanks have at least 4 BTC, scenarios initiated with minimal options:

(.venv) --> warnet run scenarios/tx_flood.py
(.venv) --> warnet run scenarios/miner_std.py --interval 5 --mature

[pinheadmz]

100 tank graph looks like this x100:

nodes:
- connect:
  - tank-0001
  - tank-0017
  - tank-0018
  - tank-0023
  - tank-0027
  - tank-0079
  - tank-0035
  - tank-0089
  image:
    tag: '27.0'
  metricsExport: true
  metrics: balance=getbalance() blocks=getblockcount()
  name: tank-0000

[pinheadmz]

After 10 minutes, every tank has at least 23 BTC. WARNET IS AWESOME!

[josibake]

now that we have a signet miner (#538) and back ported support for signet in our CVE images (#539), I'll take a stab at this. Per @pinheadmz , one idea is create a small alpine image with a pre-mined signet chain baked in. This image can be mounted as a side car to a single signet mining node, and then the rest of the nodes will sync this chain during IBD. then from there, we can run tx_flood to distribute coins (thanks for testing the numbers for tx_flood , @pinheadmz !).

this will require:

1. choosing a descriptor / challenge for this chain and keeping the descriptor xpriv private
2. premine a chain and bake it into an alpine image
3. deploying a network of non-mining nodes with with the challenge from step 1. and mount the image from step 2. to the mining node

[pinheadmz]

the image mine as well also have a bitcoin.conf with the challenge, and then it mine as well also be deployed to every tank instead of IBD, I'm not sure what the drawback of that would be, the image should cache on the first install anyway

[josibake]

started https://github.com/bitcoin-dev-project/warnet/pull/555 as a POC, was pleasantly surprised at how simple it was and how its almost entirely a change to the helm chart templates (i.e., no extra docker images to manage).

The tricky bit is where to store the files externally, but if we following the url structure of https://storage/<signetchallenge>/blocks.tar.gz, doesn't really matter where it lives.

In the PR, I also include the wallets dir, so a clever person could get the url, unzip and get the wallet file. But for this, we can just add a passphrase to the wallet and the passphrase can be in a secret config when starting the signet miner.

Would love to hear if anyone disagrees with this approach, otherwise I'll try to finish it up tomorrow with docs and an end to end example.

[willcl-ark]

Closed by #555