spike(miners): Blacklisting Validators

[Luka-Loncar]

Malicious validators could claim rewards on the subnet - we need a mechanism in place that allows miners to forbid requests to such validators.

We currently have no blacklist requirements for the subnets so every request gets processed. This means that it is easy for the subnet to be gamed and its easy for malicious validators to game rewards on the subnet.

We want to explore and understand how we can protect the miners, by having for instance a blacklist.

This card is to find all the relevant information to allow us implementing a safeguard mechanism to prevent that.

Acceptance Criteria

• [x] Design Blacklist Functionality
  • Design the blacklist functionality, including how it will filter out calls from nodes without a vpermit.
• [x] We have the relevant implementation cards

[grantdfoster]

Blacklist Implementation Spike

Issue

Current logic in main does not implement even basic blacklisting functionality, and we are getting warnings in the miner console that we are accepting requests from any neuron. There is a basic blacklist function in the template, but it is not implemented.

Solution

CLI Params

I was able to toggle on vpermit checks (i.e. enforcing only calls from validators, or neurons w/ a valid permit) by passing --blacklist.force_validator_permit to the miner run call:

python neurons/miner.py --blacklist.force_validator_permit

this runs the check self.metagraph.validator_permit[uid] using the synapse requester's uid, and if false, blacklists the request (returns the synapse, without any response.)

Implementation

the last step is actually calling this function. in neurons/miner.py's forward function, adding the following implements the blacklist function already present in the template:

[blacklisted, reason] = await self.blacklist(synapse)
if blacklisted:
    bt.logging.warning(f"Blacklisting un-registered hotkey for reason: {reason}")
    return synapse

[grantdfoster]

Update

• With basic blacklisting implemented, we can add additional layers / borrow from the example above. One obvious one is to check a validator's stake and only allow those above a certain threshold. More advanced logic could be an exploiters list, though we may want to flush out the reward system more before defining what an exploit looks like

[hide-on-bush-x]

@grantdfoster found this message on discord https://discord.com/channels/799672011265015819/1185617142914236518/1212179685895704656

Seems like a good place to do some investigation, probably the staked amount is a good indicator that someone is a validator rather than the vpermit

[hide-on-bush-x]

Found this docs useful https://docs.bittensor.com/subnets/register-validate-mine#calculate-tao-required It looks like there is an amount of TAO that depends on the subnet to become a validator and have a vpermit Could be that in our case that number is so low our miners are getting vpermit too

UPDATE: based on those docs, until we have 64 registered nodes the number will be 0, after that, in order to get a vpermit you will need to stake more than the 64th validator in order to get a vpermit

[teslashibe]

Updated my findings to here and moved out of description for clarity

Research Notes on Blacklist Logic

Code Reference: SN27 GitHub Repository

The blacklist logic is designed to filter out requests based on certain criteria to protect the miner from potentially harmful or unwanted interactions. Here’s a concise explanation of the blacklist logic:

1. Initialization: The blacklist and whitelist are initialized in the init_black_and_white_list method. This includes setting up lists for blacklisted hotkeys and coldkeys, whitelisted hotkeys and coldkeys, and a special set for exploiters' hotkeys if the configuration allows for blacklisting exploiters.

2. Blacklist Checks: The core of the blacklist logic is implemented in the base_blacklist method, which is used by specific blacklist functions for different types of requests (e.g., blacklist_specs, blacklist_allocate, blacklist_challenge). This method performs several checks to decide whether a request should be ignored:

   • Whitelist Check: If a whitelist of hotkeys is defined and the requester's hotkey is not on it, the request is allowed through, bypassing further blacklist checks.
   • Recognition Check: If the requester's hotkey is not recognized (i.e., not part of the metagraph's hotkeys), the request is blocked.
   • Stake Check: If the requester's stake is below a certain threshold (validator_permit_stake) and the configuration does not allow for whitelisting entities with insufficient stake, the request is blocked.
   • Explicit Blacklist Check: If the requester's hotkey is explicitly listed in the blacklist, the request is blocked.
   • Version Check: If there's a version whitelist (whitelist_hotkeys_version) and the requester's hotkey is not on it, the request is blocked. This is used to ensure that only entities running a compatible or approved version of the software can interact.
   • Exploiter Check: If the requester's hotkey is in the set of suspected exploiters' hotkeys, the request is blocked.

3. Priority Handling: Alongside the blacklist logic, there's also a priority system implemented in methods like base_priority, which assigns a priority to requests based on the requester's stake. This is not directly part of the blacklist logic but works in tandem with it to ensure that even allowed requests are processed in an order that reflects their importance or value to the network.

4. Dynamic Updates: The blacklist and whitelist can be dynamically updated based on the network's state and external inputs (e.g., fetching the latest version for the version check).

[teslashibe]

[mudler]

Closing this as we have follow-up cards:

• 87

• 88

• 89

• 91