spike: Research Bittensor Hyperparameters

[Luka-Loncar]

• [ ] https://github.com/orgs/masa-finance/projects/7/views/1?pane=issue&itemId=66924668

Problem:

We do not understand which hyperparameters need setting for a subnet in order to go to production in July

Objective:

Investigate and understand the purpose, usage, and impact of all of Bittensor hyperparameters

Bittensor documentation: https://docs.bittensor.com/subnets/subnet-hyperparameters

Acceptance Criteria:

• [x] Research is conducted for each of the specified hyperparameters.
• [x] Findings are documented, including the purpose, usage, and impact of each hyperparameter.
• [x] Recommendations for optimal settings are provided based on the research.
• [x] A summary of the findings is compiled, discussing potential trade-offs and considerations.
• [x] Create EPIC and tickets for setting hyperparameters
  • [x] Devnet
  • [x] Testnet
  • [ ] Main

This SPIKE ticket will help gain a deeper understanding of these crucial Bittensor hyperparameters, enabling informed decisions when configuring and optimizing the Bittensor subnet.

[hide-on-bush-x]

Analysis of Hyperparameters

1. min_allowed_weights

• Purpose and Impact: Ensures validators set weights on a minimum number of UIDs, promoting balanced resource distribution and reducing the risk of validators focusing on a small subset of miners.
• Performance and Security: A higher value improves network performance and security by encouraging interaction with more miners.
• Optimal Setting: Typically around 8.
• Current Values:
  • Localnet: Variable
  • Testnet: 1
  • Mainnet: 8
• Update Example:

  btcli sudo set --param min_allowed_weights --value 8 --netuid <NETUID>

2. max_weight_limit

• Purpose and Impact: Defines the highest weight assignable by a validator, controlling validator influence over miners.
• Performance and Security: Prevents excessive influence, ensuring an even distribution of tasks and rewards, mitigating Sybil attack risks.
• Optimal Setting: Often around 455.
• Current Values:
• Localnet: Variable
• Testnet: 65535
• Mainnet: 455
• Update Example:

btcli sudo set --param max_weight_limit --value 455 --netuid <NETUID>

3. immunity_period

• Purpose and Impact: Duration (in blocks) during which new miners or validators are protected from deregistration.
• Performance and Security: Provides new nodes time to stabilize, ensuring fair participation and network stability.
• Optimal Setting: Commonly set to about 7200 blocks (24 hours).
• Current Values:
• Localnet: Variable
• Testnet: 1000 blocks
• Mainnet: 7200 blocks
• Update Example:

btcli sudo set --param immunity_period --value 7200 --netuid <NETUID>

4. activity_cutoff

• Purpose and Impact: Number of blocks after which a validator is considered inactive if no weights are set.
• Performance and Security: Ensures only active validators contribute to the consensus, maintaining network efficiency.
• Optimal Setting: Around 5000 blocks.
• Current Values:
• Localnet: Variable
• Testnet: 5000 blocks
• Mainnet: 5000 blocks
• Update Example:

btcli sudo set --param activity_cutoff --value 5000 --netuid <NETUID>

5. tempo (not editable)

• Purpose and Impact: Specifies the cadence of subnet updates in blocks, impacting the frequency of consensus and emissions transfers.
• Performance and Security: Determines how often subnet events occur, affecting overall network dynamics.
• Optimal Setting: Typically set to 99 or 360 blocks.
• Current Values:
• Localnet: Variable
• Testnet: 360 blocks
• Mainnet: 99 blocks
• Update Example:

btcli sudo set --param tempo --value 99 --netuid <NETUID>

6. adjustment_alpha (not editable)

• Purpose and Impact: Controls the registration adjustment interval, balancing registration burn and PoW cost.
• Performance and Security: Smoother adjustment reduces registration cost thrashing.
• Optimal Setting: Commonly set to 0.97.
• Current Values:
• Localnet: Variable
• Testnet: 0.97
• Mainnet: 0.97
• Update Example:

btcli sudo set --param adjustment_alpha --value 0.97 --netuid <NETUID>

7. adjustment_interval (not editable)

• Purpose and Impact: Number of blocks after which the recycle register cost and PoW difficulty are recalculated.
• Performance and Security: Adjusts costs to align with network demand, preventing excessive or insufficient registrations.
• Optimal Setting: Set to 112 blocks.
• Current Values:
• Localnet: Variable
• Testnet: 360 blocks
• Mainnet: 112 blocks
• Update Example:

btcli sudo set --param adjustment_interval --value 112 --netuid <NETUID>

8. weights_version

• Purpose and Impact: Indicates the required minimum version of the validator code, ensuring all validators use consistent software.
• Performance and Security: Promotes network compatibility and security by standardizing validator software.
• Optimal Setting: Based on the latest stable version.
• Current Values:
• Localnet: Variable
• Testnet: 0
• Mainnet: 2013
• Update Example:

btcli sudo set --param weights_version --value 2013 --netuid <NETUID>

9. weights_rate_limit

• Purpose and Impact: Defines how often a validator can set weights on the blockchain, in blocks.
• Performance and Security: Prevents frequent weight changes, reducing chain bloat and ensuring consistent weight setting.
• Optimal Setting: Typically set to 100 blocks.
• Current Values:
• Localnet: Variable
• Testnet: 100 blocks
• Mainnet: 100 blocks
• Update Example:

btcli sudo set --param weights_rate_limit --value 100 --netuid <NETUID>

Summary

Adjusting these hyperparameters involves balancing performance, security, and fairness. Higher min_allowed_weights and max_weight_limit promote balanced validator activity, while immunity_period and activity_cutoff ensure stability and active participation. Optimal settings vary by subnet needs, and careful monitoring post-adjustment is crucial for maintaining desired network dynamics.

[teslashibe]

@hide-on-bush-x it looks like hyperparameters for subnets are published here under TAO Stats for each subnet:

1. https://x.taostats.io/subnet/13#hyperparams
2. https://x.taostats.io/subnet/2#hyperparams

[grantdfoster]

Analysis of Hyperparameters Continued...

Adding the rest of the editable params, as outlined in this file

10. serving_rate_limit

• Purpose and Impact: Determines how often you can change your node's IP address on the subtensor, expressed in blocks. Applies to both miners and validators.
• Performance and Security:
• Optimal Setting: Typically set to 100 blocks
• Current Values:
  • Localnet:
  • Testnet:
  • Mainnet:. 50
• Update Example:

  btcli sudo set --param serving_rate_limit --value 50 --netuid <NETUID>

11. min_difficulty

• Purpose and Impact:
• Performance and Security:
• Optimal Setting:
• Current Values:
  • Localnet:
  • Testnet:
  • Mainnet: 4611686018427387903
• Update Example:

  btcli sudo set --param min_difficulty --value 4611686018427387903 --netuid <NETUID>

12. max_difficulty

• Purpose and Impact:
• Performance and Security:
• Optimal Setting:
• Current Values:
  • Localnet:
  • Testnet:
  • Mainnet: 4611686018427387903
• Update Example:

  btcli sudo set --param max_difficulty --value 4611686018427387903 --netuid <NETUID>

13. network_registration_allowed

• Purpose and Impact:
• Performance and Security:
• Optimal Setting:
• Current Values:
  • Localnet:
  • Testnet:
  • Mainnet: True
• Update Example:

  btcli sudo set --param network_registration_allowed --value true --netuid <NETUID>

14. network_pow_registration_allowed

• Purpose and Impact:
• Performance and Security:
• Optimal Setting:
• Current Values:
  • Localnet:
  • Testnet:
  • Mainnet: True
• Update Example:

  btcli sudo set --param network_pow_registration_allowed --value true --netuid <NETUID>

15. min_burn

• Purpose and Impact:
• Performance and Security:
• Optimal Setting:
• Current Values:
  • Localnet:
  • Testnet:
  • Mainnet: 1000000000
• Update Example:

  btcli sudo set --param min_burn --value 1000000000 --netuid <NETUID>

16. max_burn

• Purpose and Impact:
• Performance and Security:
• Optimal Setting:
• Current Values:
  • Localnet:
  • Testnet:
  • Mainnet: 100000000000
• Update Example:

  btcli sudo set --param max_burn --value 100000000000 --netuid <NETUID>

[H34D]

Summary

Adjusting these hyperparameters involves balancing performance, security, and fairness. Higher min_allowed_weights and max_weight_limit promote balanced validator activity, while immunity_period and activity_cutoff ensure stability and active participation. Optimal settings vary by subnet needs, and careful monitoring post-adjustment is crucial for maintaining desired network dynamics.

i am assuming they must be set by the subnet owner account / role. right?

[mudler]

seconding @H34D thoughts - few Q pops up in my head here:

• where are we going to set those hyperparameters to take effect?
• the user must be aware of these settings? shall we expose it to the user and be documented?

[hide-on-bush-x]

@H34D Yes, this settings are set by the owner if the subnet @mudler we should start setting this params in one of our devnet subnets as a first experiment This hyperparameters can be seen by the user using the CLI

something like

btcli subnet hyperparameters --netuid 1 --subtensor.chain_endpoint  ws://54.205.45.3:9945

   Subnet Hyperparameters - NETUID: 1 - unknown
 HYPERPARAMETER              VALUE
   rho                       10
   kappa                     32767
   immunity_period           5000
   min_allowed_weights       1
   max_weight_limit          65535
   tempo                     360
   min_difficulty            18446744073709551615
   max_difficulty            18446744073709551615
   weights_version           0
   weights_rate_limit        100
   adjustment_interval       360
   activity_cutoff           5000
   registration_allowed      True
   target_regs_per_interval  1
   min_burn                  1
   max_burn                  100000000000
   bonds_moving_avg          900000
   max_regs_per_block        1
   serving_rate_limit        50
   max_validators            64
   adjustment_alpha          17893341751498265066
   difficulty                18446744073709551615

I will test weights_rate_limit by setting weights for a set of miners and then removing the rate limit after I get the error about "setting weights too fast" ( Will post my findings here )

[mudler]

@H34D Yes, this settings are set by the owner if the subnet @mudler we should start setting this params in one of our devnet subnets as a first experiment This hyperparameters can be seen by the user using the CLI

something like

btcli subnet hyperparameters --netuid 1 --subtensor.chain_endpoint  ws://54.205.45.3:9945

   Subnet Hyperparameters - NETUID: 1 - unknown
 HYPERPARAMETER              VALUE
   rho                       10
   kappa                     32767
   immunity_period           5000
   min_allowed_weights       1
   max_weight_limit          65535
   tempo                     360
   min_difficulty            18446744073709551615
   max_difficulty            18446744073709551615
   weights_version           0
   weights_rate_limit        100
   adjustment_interval       360
   activity_cutoff           5000
   registration_allowed      True
   target_regs_per_interval  1
   min_burn                  1
   max_burn                  100000000000
   bonds_moving_avg          900000
   max_regs_per_block        1
   serving_rate_limit        50
   max_validators            64
   adjustment_alpha          17893341751498265066
   difficulty                18446744073709551615

I will test weights_rate_limit by setting weights for a set of miners and then removing the rate limit after I get the error about "setting weights too fast" ( Will post my findings here )

Cool - can you create the relevant follow-up card for the implementation, pointing out where the the changes should happen in the codebase ? We are going to pick that up during planning then - Thanks!

[hide-on-bush-x]

Sure, will do 🫡