Weighted sample of quality + period scores composite approach with softmax by Sergiy Pavlyuk

[spavlyuk80]

Task statement: Improve the selection process for miners, to get the best product possible for the frontend, whilst also making sure we have enough data on each miner to judge them.

Before considering the solutions, let's consider both, validator and miner interests/incentives/constraints.

Validators:

• effective use of traffic allocation (consequence of fair use and miner's prioritisation mechanism)
• maximise alignment with consensus (maximise rewards, reputation, etc)
• maintain network fairness: each request should be assigned to a miner randomly, ensuring that all miners have an equal chance of receiving any given request. This is from the docs, but my understanding is that task assignment chance must be greater than 0.0
• collect enough data (through either organic or synthetic queries) to be able to fairly assess miner's for the future

Miner:

• maximise incentive (points) per task. Roughly max of "best in the crowd" and/or "doing a job no one else wants to do"
• maintain "reputation": do not get kicked out
• From the prioritisation mechanism white paper: a miner only has to maximise the score at the next scoring period
• miner has a choice to not respond or may not respond (internal server error or too many requests error)

Both players are interested in maximising score with some constraints in capacity and fairness and more importantly maintaining some randomness.

Design requirements:

• scalable
• keep it simple, it is first iteration only)
• edge cases handling

Proposed implementation

Use weighted sample approach + softmax (cool feature of this function is the temperature parameter that 'blurs' probabilities and increases non-zero chance of being selected from a random sample). Use available info(average_quality and period_scores) from the last contender records. Quality scores are sticky - so one data point is enough. All contenders start with empty period score in each cycle, thus everyone has the same chances, but things might chance during the cycle. How period score works is explained here

Not technical explanation

we calculate a composite score based on two parameters for each contender: average_quality_score and current cycle period_score

Composite score is calculated as:

Composite score = WEIGHT_QUALITY_SCORE normalised average_quality_score + WEIGHT_PERIOD_SCORE normalised_period_score

where:

• WEIGHT_QUALITY_SCORE (current value 1.0) - weight of average quality score
• WEIGHT_PERIOD_SCORE (current_value 0.7) - weight of current period score

Therefore composite score favours quality score with current settings.

In order to increase chances of low score contender to be selected, we calculate probabilities of selection using softmax function with high temperature. Temperature is set by SOFTMAX_TEMPERATURE (current value 2.0) - smoothing parameter for softmax. Temperature of 2.0 effectively increases the chances of low score contender to be selected for query.

Output of softmax is probability of each contender being selected (sum of probabilites = 1).

We then apply weighted sampling to select top_x contenders (current default value = 5, set in select_contenders function arguments.

A combination of softmax and weighted sampling ensures that even the lowest score contender has a chance for being selected for query.

Data requirements

No need for new data and complex feature engineering. Fetch all needed info from the contenders table, augment it with score from stats table. No need for indices as tables are only of len(all contenders), but to be safe return latest from stats table.

Benefits of this approach:

• easy to implement
• does not rely on a lot of historical data
• easy to scale while we watch table sizes
• easy to explain to business

  Drawbacks:

• use of somewhat arbitrary weights and softmax params. But need to look at statistics

Expected result: somewhat novel and more fair approach to select contenders for query, while ensuring we still collect information from all the contenders.

Further improvements:

• separate weights and logic for organic and synthetic queries
• consider a cut-off score to limit contenders with bad historical score all of this however goes against the principle of maintaining network fairness for validator

Some Notes:

• contenders table is updated in the beginning of each cycle and some of the info is set to zero (https://github.com/namoray/nineteen/blob/0a9ebe41b39cc1f8a1787c7670b3305572e3336f/validator/control_node/src/cycle/refresh_contenders.py#L147), some info will have to be queried from historical events...
• if the contender was not selected, he will be pushed with zero values to history - also will work with the model
• I noticed that the source code actually uses non linear transformations instead of weights, I think my approach is more flexible

Fixes: #70 seasoning?

[spavlyuk80]

This solution is perhaps on the right lines, but has a fundamental issue.

I do not want to serve new users to the frontend.

This solution ensures we direct organic requests to the worst miners with non zero probability, which is the opposite of what we want to do. We also use the combined quality score, which is not the right metric as it doens't take into account speed. Normalised net score is the right metric.

Yep, now I see that period score only includes rate limiters. thanks

In regard to the solution. You can control the chance of querying the worst miner by softmax temperature. Thus, the next iteration would be to use the same function for synthetic and organic queries with different temperatures (high for organic, low for synthetic).

[spavlyuk80]

Fixes discussed above - always prefer to finish tasks)