Modularization of Miner/Validators tasks

[juanmanso]

Refactor: Modularisation of tasks

Problem

Scaling more endpoints will make code complex, before we add more endpoints we will need to refactor this.

🎯 Goal

As we grow the codebase of the masa-bittensor repo, we write this RFC to document and open the conversation on the problematic that rose while integrating the Discord Profile and Web Scraper tasks to the Subnet.

🥅 What we want to achieve

• Localisation of development
• Ease on reviews
• Minimize feature’s footprint across the codebase
• Ensure stability of environment through decoupling
• Scalable solution
• Black box behavior of Validator/Miner tasks to the infra

---

🪨 Current state

Highest level

The repo has two main folders:

• masa
• neurons

neurons implement the Miner and Validator classes which extended our masa custom implementation for usage. These are our entry point scripts for them to run.

Masa's Subnet Logic

Inside masa folder we can find most importatly:

• api
  • Defines Synapse Object
  • Defines Validator's FastAPI
• base
  • Defines building blocks to be extended (Neuron, Miner & Validator)
• miner
  • Defines Masa Protocol Request Class
  • Depending the required task to perform, it extends and implements custom class
• validator
  • Implements base forwarder class
  • Depending the required task to perform, it extends the forwarder class and implements rewards and parser functions

⚠️ Problem

Everytime we want to add a new task, we need to:

• Create a new Miner Class extending Masa Protocol's Request
• Define new types for query params and request responses
• Add optional attribute to Synapse Object on request.py to receive query or body params
• Add new api route on validator_api.py and function to be called using custom Forwarder
• Implement custom rewards and parser (this one based on the return type of the response)

‼️ On paper, that seems pretty easy to achieve but in reality code is hard to follow and developers need to jump around files a lot to even understand how things are connected and working.

Additionally, since the forwarder class is pretty much fixed in the steps it follows, custom behavior for each task is hard to achieve

---

🔬 Proposed solution

TL;DR:

• services folder
  • has 1 request class per external service to the network
• utils folder
  • generic and deterministic uid fetchers (i.e. get_random_uids)
• api
  • validator_api which sets route and imports function to run upon request (from task)
• base
  • shared methods and attributes across neurons, miners and validators
    • Neuron's state management
    • Neuron's metagraph management
    • Validator's update weights logic
    • etc.
• Neurons
  • miner entry point for users that want to run miner node
  • validator entry point for users that want to run a validator node
• tasks folder (with subfolders, 1 for each):
  • Custom Synapse object → protocol.py
  • Custom types for the whole process (responses, query, etc.) → types.py
  • Miner logic → miner.py
    • Extends whichever Request class it needs (1 Request Class per External Service)
    • Process that request as required by task:
      • Extra calculations
      • Format to match Synapse response
  • Validator logic → validator.py
    • def forward()
      • Which miners (uids) to query based on its task
      • Perform synapse to miners
      • Handle responses
      • Score responses and update uid scores
      • Return actual respond to user
    • def reward()
      • how validate and score responses
    • def generate_response()
      • Based on responses and scores, generate final output for user

[juanmanso]

First version of RFC is ready for comments:

• https://www.notion.so/masafinance/Refactor-Modularisation-of-tasks-40350e2d36cb461ab47d6544974bfee2?pvs=4

TL;DR of proposal:

- services folder
    - has 1 request class per external service to the network
- utils folder
    - generic and deterministic uid fetchers (i.e. get_random_uids)
- api
    - validator_api which sets route and imports function to run upon request (from task)
- base
    - shared methods and attributes across neurons, miners and validators
        - Neuron's state management
        - Neuron's metagraph management
        - Validator's update weights logic
        - etc.
- Neurons
    - miner entry point for users that want to run miner node
    - validator entry point for users that want to run a validator node
- tasks folder (with subfolders, 1 for each):
    - Custom Synapse object → `protocol.py`
    - Custom types for the whole process (responses, query, etc.) → `types.py`
    - Miner logic → `miner.py`
        - Extends whichever Request class it needs (1 Request Class per External Service)
        - Process that request as required by task:
            - Extra calculations
            - Format to match Synapse response
    - Validator logic → `validator.py`
        - def forward()
            - Which miners (uids) to query based on its task
            - Perform synapse to miners
            - Handle responses
            - Score responses and update uid scores
            - Return actual respond to user
        - def reward()
            - how validate and score responses
        - def generate_response()
            - Based on responses and scores, generate final output for user

[juanmanso]

Moving it to review and getting to tinker the approach to share some snippets/env for review if that's eventually needed

[juanmanso]

Refactor: Modularisation of tasks

🎯 Goal

As we grow the codebase of the masa-bittensor repo, we write this RFC to document and open the conversation on the problematic that rose while integrating the Discord Profile and Web Scraper tasks to the Subnet.

🥅 What we want to achieve

• Localisation of development
• Ease on reviews
• Minimize feature’s footprint across the codebase
• Ensure stability of environment through decoupling
• Scalable solution
• Black box behavior of Validator/Miner tasks to the infra

---

🪨 Current state

Highest level

The repo has two main folders:

• masa
• neurons

neurons implement the Miner and Validator classes which extended our masa custom implementation for usage. These are our entry point scripts for them to run.

Masa's Subnet Logic

Inside masa folder we can find most importatly:

• api
  • Defines Synapse Object
  • Defines Validator's FastAPI
• base
  • Defines building blocks to be extended (Neuron, Miner & Validator)
• miner
  • Defines Masa Protocol Request Class
  • Depending the required task to perform, it extends and implements custom class
• validator
  • Implements base forwarder class
  • Depending the required task to perform, it extends the forwarder class and implements rewards and parser functions

⚠️ Problem

Everytime we want to add a new task, we need to:

• Create a new Miner Class extending Masa Protocol's Request
• Define new types for query params and request responses
• Add optional attribute to Synapse Object on request.py to receive query or body params
• Add new api route on validator_api.py and function to be called using custom Forwarder
• Implement custom rewards and parser (this one based on the return type of the response)

‼️ On paper, that seems pretty easy to achieve but in reality code is hard to follow and developers need to jump around files a lot to even understand how things are connected and working.

Additionally, since the forwarder class is pretty much fixed in the steps it follows, custom behavior for each task is hard to achieve

---

🔬 Proposed solution

TL;DR:

• services folder
  • has 1 request class per external service to the network
• utils folder
  • generic and deterministic uid fetchers (i.e. get_random_uids)
• api
  • validator_api which sets route and imports function to run upon request (from task)
• base
  • shared methods and attributes across neurons, miners and validators
    • Neuron's state management
    • Neuron's metagraph management
    • Validator's update weights logic
    • etc.
• Neurons
  • miner entry point for users that want to run miner node
  • validator entry point for users that want to run a validator node
• tasks folder (with subfolders, 1 for each):
  • Custom Synapse object → protocol.py
  • Custom types for the whole process (responses, query, etc.) → types.py
  • Miner logic → miner.py
    • Extends whichever Request class it needs (1 Request Class per External Service)
    • Process that request as required by task:
      • Extra calculations
      • Format to match Synapse response
  • Validator logic → validator.py
    • def forward()
      • Which miners (uids) to query based on its task
      • Perform synapse to miners
      • Handle responses
      • Score responses and update uid scores
      • Return actual respond to user
    • def reward()
      • how validate and score responses
    • def generate_response()
      • Based on responses and scores, generate final output for user

[juanmanso]

Update on the code spike: https://masafinance.slack.com/archives/C0742F6KXB8/p1718194559616379

[juanmanso]

Closed since follow-up actions (issues) are defined and created.

Next step is set on masa-finance/bt-refactor-spike#1