Support for updating chain parameters using xfield

[azuchi]

Tapyrus block headers currently support the following xfield types:

• 1: Aggregate public key.
• 2: Max block size

Add to Tapyrus Signer the ability to generate new blocks by specifying these xfield types. The basic flow is as follows:

1. First, for each Signer node, register the xfield change by specifying the xfield type and value to be changed and the block height to include it (these data should be persistent). It is necessary to specify a block height with sufficient margin to allow other nodes to finish their operations.
2. The master node creating the candidate block to apply the change if it has an outstanding xfield type change and its block height is greater than or equal to the block to be created (specify them in the tapyrus core RPC).
3. When a node receives a candidate block from the master node, if the candidate block contains a change of xfield type, it checks to see if the change has been registered on its own node.
   • If so, it approves it and does nothing.
   • If not, it treats the candidate block as invalid.
4. If a signature can be generated for the candidate block for the threshold and the block is complete, the block that updates the xfield type is successfully generated.

In addition to above, add a CLI command to delete registered xfield information (to delete data registered in error).

[Naviabheeman]

PR #167 adds federation command. Some questions at this point to discuss are:

1. Where to write the federation change file? can we keep it as a fixed default directory and document it or make it configurable?
2. All register and unregister commands change the same file or create multiple files ( which is the current behaviour)?

[azuchi]

Where to write the federation change file? can we keep it as a fixed default directory and document it or make it configurable?

Let $HOME/.tapyrus-signer be the base data directory. The data directory can be changed using the configuration option --datadir. This setting can be specified as a command line argument or in a toml file like --log-file option.

All register and unregister commands change the same file or create multiple files ( which is the current behaviour)?

I think it's fine to use the same file. Does it need to be separated?

[Naviabheeman]

I think it's fine to use the same file. Does it need to be separated?

@azuchi My thought process is that we can make each file readonly so that we can ensure that these files are not accidentally/intentionally modified. I would also want to add an aggregate signature to make sure the file cannot be modified.

In the case of aggregated public key it's not possible to generate the new public key without the involvement of all the signers. The protocol ensures that the change is genuine. But in the case of max block size, it is possible for anyone to write this kind of file in all the signer nodes. If such a change can make the block size very small the whole network would choke. All signers would sign the block assuming that the file in its $HOME is genuine. But it would result in the whole Tapyrus-core network decreasing the block size and transactions being rejected because with less throughput mempools would be full.

If not a full aggregated signature, we need some scheme to ensure that all xfield changes are only through 'federation change' command and are intentional.

[Naviabheeman]

@azuchi Signer code has logic to support multiple federations as it is. The master node does getnewblock, verify_block and add_aggregated_public_key_if_needed for each block. When federations.toml is extended with multiple federation definitions this code can make the aggregated pub key change in the block header before signing the block. This is not verified by other signers though.

I'm implementing 'federation' command like this:

• write all updates to one file, federationchange.dat. I'm treating this file as a list of pending changes. So when an xfield change is successful, it is removed from the file.
• The signer node would monitor this file and sync the changes to an in memory list.
• when the block height is reached, it adds the xfield to the block.
• remove that completed entry from federationchange.dat if the xfield change was successful in the that round
• When the block fails, as federationchange.dat file is available to the next master node it would be retried.

1. If we add threshold and node-vss to this command, we can make 'federation' command write to two files, federations.toml and federationchange.dat. i.e. change in aggregated-pub-key goes to federations.toml and change in max-block-size and any new xfield can go to federationchange.dat. Then running one command on all signer nodes will be enough to change any xfield. But the command is not as friendly as editing the toml.

2. we can use federations.toml as a reference for the federation command. The command would verify if the federation details, threshold and node-vss are added to federations.toml. Check their validity and then only add aggregated pub key to read from federationchange.dat to the block. In this case we need to update federations.toml before aggregate-pub-key change.

3. we can simplify all of the above and use just federations.toml for all xfield changes. A thread in the signer node can monitor the file and synch it to the list in memory. In this strategy we don't even need a command to register the change. But we need to make it safe somehow. It has an advantage that one file update can manage any xfield change. Which of these would be preferable?

[azuchi]

Signer code has logic to support multiple federations as it is. The master node does getnewblock, verify_block and add_aggregated_public_key_if_needed for each block.

Yes. The master node switches in a round-robin fashion. The minimum number of federations is 3. For now, we assume that the federation is composed of members in a trusting relationship. In other words, updating the aggregate public key(updating the federation members) must be agreed upon in the real world.

I'm treating this file as a list of pending changes.

If a command is executed multiple times and multiple xfield changes occur, are they all recorded?

we can simplify all of the above and use just federations.toml for all xfield changes.

In this case, you mean all changes are directly editing the file?

[Naviabheeman]

If a command is executed multiple times and multiple xfield changes occur, are they all recorded?

The file is guarded by a lock to protect it within a process. And I added a test to run two commands in parallel.
It does not cause any failure yet. This conversation has given me an idea to extend the test i.e to repeat the command with a new block height and verify that all writes were recorded. Otherwise we need to go back to multiple file outputs for each command execution described in this comment

In this case, you mean all changes are directly editing the file?

Yes. I mean edit the file directly. I would also like to add a verification field like signature for each change. If the whole federation was involved in making the decision to change the parameter the signature can be generated for all xfields like aggregated public key.

[azuchi]

Yes. I mean edit the file directly.

I see. Let's implement this(method 3) instead of the command first. Also, I don't think you need to monitor the file, just load it at startup.

[Naviabheeman]

I see. Let's implement this(method 3) instead of the command first.

Do you mean with a verifiable signature or without it? Probably this:

[[xfields]]
{
block_height = <bock_height>
<xfield_name> = <xfield_value>
signature = <signature>
}

Also, I don't think you need to monitor the file, just load it at startup.

Sure.

[azuchi]

Do you mean with a verifiable signature or without it? Probably this:

If you add a signature;

• Is this required not only for aggregate public key updates, but also for updates of other xfield types(like max block size)?
• What message and public key is this signature for?

[Naviabheeman]

• Is this required not only for aggregate public key updates, but also for updates of other xfield types(like max block size)?

I think we need the signature for all xfield changes other than aggregated-pub key. In the case of aggregated pub key change, the node vss needs to be generated with the cooperation of all or majority of signers. Other xfields like maxblocks size can be added by anyone with access to the file. we can have a threshold signature, similar to block proof. i.e. sign the hash(serialized_xfield) using the distributed signature scheme. This will ensure that all signers are aware of the change in xfield.

So the signer implementation would:

• read federations.toml
• verify the signature in each xfield change using the aggregated pub key of the federation at that block height.
• while generating the block with xfield, master node can add the xfield to block header.
• other nodes participating in the block generation can verify the xfield in their federation.toml, verify its signature and then provide the partial signature to the block.

• What message and public key is this signature for?

sign the hash(serialized_xfield) using the distributed signature scheme. and verify using aggregated-pub key at that block-height.

[azuchi]

I see. Let's add a signature field.

verify the signature in each xfield change using the aggregated pub key of the federation at that block height.

for example, when updating a new aggregate public key with a block height of 100, signature public key is the aggregate public key before the update, right?

[Naviabheeman]

Earlier I meant that we don't need a signature for aggregated pub key change as their node vss can ensure it is secure. But on the other hand it's not uniform. So I think yes, we can add a signature to aggregated pub key change too.

for example, when updating a new aggregate public key with a block height of 100, signature public key is the aggregate public key before the update, right?

Yes, the signature pub key is the aggregate public key before the update.

[Naviabheeman]

169 and #170 add the needed support.

TODO: Add more tests - multiple federations with multiple federation-changes