Score statistics processor needs to process PP for imported highscores

[smoogipoo]

When we deploy lazer for ranking purposes, we need to completely shut down osu-performance and have PP be solely updated by osu-queue-score-statistics. Otherwise, because osu-performance only reads from the legacy tables, it will exclude any scores set in lazer.

But right now all imported highscores bypass osu-queue-score-statistics so it'll miss any scores set in osu!stable.

[bdach]

I started looking into this, but I'm not sure of a few things:

• During the import process, should the imported scores be enqueued onto redis for processing, or processed inline? I'm slightly worried about the backpressure on the queue processing while the import is running - namely the possibility of the import process spraying scores onto the queue faster than the score processor can go through them.

• As I understand it, ScorePerformanceProcessor is the one processor that must definitely run for correct operation, but how about UserTotalPerformanceProcessor? In theory, the pp from any imported score should already be reflected in the user's pp total, but maybe it's not?

  Another thing that worries me here is reprocesses due to the upgrade of ScoreStatisticsQueueProcessor.VERSION - is that something we realistically do? I recall something being said about the "revert" operation of processors being basically unused, but is it?

[smoogipoo]

During the import process, should the imported scores be enqueued onto redis for processing, or processed inline?

Will need to confirm with @peppy, because the import process currently has a join on the score_process_queue table exactly for this - to wait for osu-performance calcs so that those are also imported into solo_scores_performance.
But I've already told him that we probably don't need to do that (if it's causing issues elsewhere), and instead run a full pp reprocess - it might just be fast enough to handle it.

For the "watch" mode of the processor, it definitely needs to happen. It should be very little load by that point.

As I understand it, ScorePerformanceProcessor is the one processor that must definitely run for correct operation, but how about UserTotalPerformanceProcessor?

Both need to run because the user total is derived from the score performance. Here's the process I imagine:

Full import:

• Scores are imported into solo_scores.
• PP is not imported into solo_scores_performance.

Post- full import:

• Run a full reprocess of PP - ScorePerformanceProcessor + UserTotalPerformanceProcessor.

Watch mode:

• Scores are imported into solo_scores.
• Event is enqueued onto the score process queue.
• Score is dequeued from the queue, and both ScorePerformanceProcessor + UserTotalPerformanceProcessor are run on it.

[bdach]

But I've already told him that we probably don't need to do that (if it's causing issues elsewhere), and instead run a full pp reprocess - it might just be fast enough to handle it.

For the "watch" mode of the processor, it definitely needs to happen. It should be very little load by that point.

I'm not sure I follow this part - the way I'm understanding it is that pp processing should be ran both inline during the import process and as items arrive in the queue? That sounds like processing the imported score multiple times which seems wasteful? In what circumstance would an imported score with pp processed inline go through the queues again - is it the version upgrade scenario?

[smoogipoo]

Explained on voice, but it's only the watch mode of the processor that needs to push to the queue.