how to calculate Trueskill socre ? thanks

[minshengxinwen]

I'm confused as to how Trueskill is calculated. How does votes.tsv get to qscores.tsv? Using the category safety as an example, where images are compared two by two, are those hundreds of thousands of images equivalent to hundreds of thousands of players? I'm confused, how does votes.tsv set the mean variance and update the scores after comparison? Read the synopsis of trueskill, but not sure how to apply it in this case. Can you give me some ideas? Or is there any code for this part, so I can see some myself how it is calculated. Thanks.

Translated with DeepL.com (free version)

[aleksandrskoselevs]

Hi. There is no official description of the dataset and what the qscores.tsv actually represent. So the following explanation is based on my understanding of the reference paper [1]

How does votes.tsv get to qscores.tsv?

A TrueSkill algorithm is run with votes.tsv as the input. As a result it produces qscores.tsv

Using the category safety as an example, where images are compared two by two, are those hundreds of thousands of images equivalent to hundreds of thousands of players?

Each row in votes.tsv is one vote in a comparison of two images, by one person, for one category. For example, a row might be one person answering the question "Which of the two images you find more safe?"

I'm confused, how does votes.tsv set the mean variance and update the scores after comparison? Read the synopsis of trueskill, but not sure how to apply it in this case. Can you give me some ideas?

Here is a snippet of my implementation:

location_ids = data['location_id']

import trueskill

# Create a TrueSkill environment
env = trueskill.TrueSkill()

# Create a dictionary to store the TrueSkill scores for each location ID
trueskill_scores = {}

# Iterate over each location ID
for location_id in location_ids:
    # Get the votes for the current location ID
    location_votes = votes_not_seen[(votes_not_seen['left_id'] == location_id) | (votes_not_seen['right_id'] == location_id)]

    # Create a TrueSkill rating for the current location ID
    rating = env.create_rating()

    # Iterate over each vote for the current location ID
    for _, vote in location_votes.iterrows():
        left_id = vote['left_id']
        right_id = vote['right_id']
        winner = vote['winner']

        # Determine the opponent ID based on the winner
        if winner == 'left':
            opponent_id = right_id
        elif winner == 'right':
            opponent_id = left_id
        else:
            continue

        # Update the TrueSkill rating based on the outcome of the vote
        if location_id == winner:
            rating, _ = env.rate_1vs1(rating, env.create_rating())
        elif location_id == opponent_id:
            _, rating = env.rate_1vs1(env.create_rating(), rating)

    # Store the TrueSkill score for the current location ID
    trueskill_scores[location_id] = rating.mu

Or is there any code for this part, so I can see some myself how it is calculated.

If there is still interest for this, I'll work on formatting and publishing my implementation for this part.

Thanks.

You're welcome :)

[1] A. Dubey, N. Naik, D. Parikh, R. Raskar, and C. A. Hidalgo, ‘Deep Learning the City : Quantifying Urban Perception At A Global Scale’, Sep. 12, 2016, arXiv: arXiv:1608.01769. Accessed: Apr. 26, 2023. [Online]. Available: http://arxiv.org/abs/1608.01769

[Filbert-0306]

Hi, I would like to know how the trueskill.score is transformed into trueskill.stds.-1. Could you please explain it to me? Thanks.

[Filbert-0306]

@minshengxinwen Thanks!