search

MaxOhn / rosu-pp-py

osu! difficulty and pp calculation for all modes
MIT License
23 stars 19 forks source link

PP calculations with acc and mod settet are inaccurate ? #5

Closed EnricoBaivo closed 1 year ago

EnricoBaivo commented 1 year ago

Tested some calculations and i guess they are not working right or i would expect a diffrence arount 5 to 9 pp but above seems unrealsitic ?

Here are the tested Maps and results:

PP is not correct percent 93.15642458100558 nearest_percent 93.0 reached_pp 95.466 nearest_expected_pp 118.82900490746322 beatmapset_id 1068768 beatmap_id 2237467 mod 0
PP is not correct percent 99.71014492753623 nearest_percent 99.5 reached_pp 87.2878 nearest_expected_pp 77.57875228446055 beatmapset_id 770668 beatmap_id 1621242 mod 0
PP is not correct percent 97.34219269102991 nearest_percent 97.5 reached_pp 81.033 nearest_expected_pp 95.85743247978336 beatmapset_id 361902 beatmap_id 794985 mod 0
PP is not correct percent 98.85057471264368 nearest_percent 99.0 reached_pp 80.1939 nearest_expected_pp 67.76165325386097 beatmapset_id 557532 beatmap_id 1182090 mod 64
PP is not correct percent 95.97701149425288 nearest_percent 96.0 reached_pp 71.5572 nearest_expected_pp 85.16137951912881 beatmapset_id 737128 beatmap_id 1555610 mod 0
PP is not correct percent 93.30218068535827 nearest_percent 93.0 reached_pp 69.4847 nearest_expected_pp 99.62012841089103 beatmapset_id 770306 beatmap_id 1619564 mod 0
PP is not correct percent 97.39229024943312 nearest_percent 97.5 reached_pp 69.0237 nearest_expected_pp 89.01452682785447 beatmapset_id 358451 beatmap_id 788219 mod 0
PP is not correct percent 92.93680297397769 nearest_percent 93.0 reached_pp 67.4181 nearest_expected_pp 87.25992782162692 beatmapset_id 1525595 beatmap_id 3126758 mod 0
PP is not correct percent 95.41160593792173 nearest_percent 95.0 reached_pp 64.6528 nearest_expected_pp 73.57476957455192 beatmapset_id 628149 beatmap_id 1324387 mod 0
PP is not correct percent 96.66666666666667 nearest_percent 97.0 reached_pp 64.3054 nearest_expected_pp 71.57815999991918 beatmapset_id 192824 beatmap_id 458405 mod 0
PP is not correct percent 95.99083619702176 nearest_percent 96.0 reached_pp 60.6873 nearest_expected_pp 69.93167029184428 beatmapset_id 689957 beatmap_id 1490190 mod 0
PP is not correct percent 92.07317073170732 nearest_percent 93.0 reached_pp 59.2302 nearest_expected_pp 78.58908144862077 beatmapset_id 720240 beatmap_id 1524133 mod 0
PP is not correct percent 95.28301886792453 nearest_percent 95.0 reached_pp 56.1488 nearest_expected_pp 67.2556782001929 beatmapset_id 1028594 beatmap_id 2150739 mod 0
PP is not correct percent 93.6842105263158 nearest_percent 93.0 reached_pp 55.3142 nearest_expected_pp 62.49122114572638 beatmapset_id 1178935 beatmap_id 2473353 mod 0
PP is not correct percent 96.88172043010753 nearest_percent 97.0 reached_pp 54.8754 nearest_expected_pp 74.70458182487181 beatmapset_id 459149 beatmap_id 983692 mod 0
PP is not correct percent 94.74412171507606 nearest_percent 95.0 reached_pp 53.115 nearest_expected_pp 59.5357568995785 beatmapset_id 1525595 beatmap_id 3126205 mod 0
PP is not correct percent 94.04761904761905 nearest_percent 95.0 reached_pp 52.5032 nearest_expected_pp 72.94102409849157 beatmapset_id 223871 beatmap_id 522773 mod 0
PP is not correct percent 94.38860971524288 nearest_percent 95.0 reached_pp 51.2112 nearest_expected_pp 83.10313494396392 beatmapset_id 43466 beatmap_id 136400 mod 0
PP is not correct percent 98.33333333333333 nearest_percent 98.5 reached_pp 50.9933 nearest_expected_pp 44.29966826547132 beatmapset_id 861685 beatmap_id 1802636 mod 64
PP is not correct percent 96.93251533742331 nearest_percent 97.0 reached_pp 50.2905 nearest_expected_pp 63.70999901714233 beatmapset_id 371128 beatmap_id 827932 mod 0
PP is not correct percent 97.67932489451476 nearest_percent 97.5 reached_pp 49.7207 nearest_expected_pp 75.37772827668594 beatmapset_id 497942 beatmap_id 1136506 mod 0
PP is not correct percent 94.51901565995526 nearest_percent 95.0 reached_pp 49.605 nearest_expected_pp 66.417379717603 beatmapset_id 596704 beatmap_id 1264070 mod 0
PP is not correct percent 88.56526429341963 nearest_percent 90.0 reached_pp 49.4296 nearest_expected_pp 68.01653433272227 beatmapset_id 584520 beatmap_id 1241396 mod 0
PP is not correct percent 95.45977011494253 nearest_percent 95.0 reached_pp 49.3795 nearest_expected_pp 72.48301498593088 beatmapset_id 112938 beatmap_id 293189 mod 0
PP is not correct percent 93.2645034414946 nearest_percent 93.0 reached_pp 48.7902 nearest_expected_pp 108.46941776660552 beatmapset_id 983911 beatmap_id 2118443 mod 0
PP is not correct percent 89.8572884811417 nearest_percent 90.0 reached_pp 48.5 nearest_expected_pp 88.65102275785704 beatmapset_id 865746 beatmap_id 1810188 mod 0
PP is not correct percent 94.15954415954417 nearest_percent 95.0 reached_pp 48.2749 nearest_expected_pp 65.10027375515762 beatmapset_id 474376 beatmap_id 1018869 mod 0
PP is not correct percent 89.11022576361222 nearest_percent 90.0 reached_pp 48.1211 nearest_expected_pp 85.24829548789538 beatmapset_id 443589 beatmap_id 953394 mod 0
PP is not correct percent 93.37748344370861 nearest_percent 93.0 reached_pp 48.0648 nearest_expected_pp 68.55196659507594 beatmapset_id 1669877 beatmap_id 3410893 mod 0
PP is not correct percent 89.54545454545455 nearest_percent 90.0 reached_pp 47.172 nearest_expected_pp 64.76375568229946 beatmapset_id 865746 beatmap_id 1810299 mod 0
PP is not correct percent 95.32908704883228 nearest_percent 95.0 reached_pp 46.878 nearest_expected_pp 75.43170377615829 beatmapset_id 386117 beatmap_id 853166 mod 0
PP is not correct percent 93.50198412698413 nearest_percent 93.0 reached_pp 46.693 nearest_expected_pp 62.47782598629197 beatmapset_id 828931 beatmap_id 1736730 mod 0
PP is not correct percent 97.16666666666667 nearest_percent 97.0 reached_pp 46.5769 nearest_expected_pp 81.96064974927766 beatmapset_id 431220 beatmap_id 1027645 mod 0
PP is not correct percent 91.34897360703812 nearest_percent 90.0 reached_pp 46.3384 nearest_expected_pp 75.20604381223603 beatmapset_id 337527 beatmap_id 746736 mod 0
PP is not correct percent 94.51754385964912 nearest_percent 95.0 reached_pp 46.185 nearest_expected_pp 76.7624432603595 beatmapset_id 426992 beatmap_id 921857 mod 0
PP is not correct percent 94.35411020776876 nearest_percent 95.0 reached_pp 46.038 nearest_expected_pp 74.6823206966474 beatmapset_id 56633 beatmap_id 171126 mod 0
PP is not correct percent 87.6750700280112 nearest_percent 90.0 reached_pp 45.496 nearest_expected_pp 70.89164273168481 beatmapset_id 1073074 beatmap_id 2245784 mod 0
PP is not correct percent 88.0 nearest_percent 90.0 reached_pp 44.5204 nearest_expected_pp 114.96614182830966 beatmapset_id 962088 beatmap_id 2014469 mod 0
PP is not correct percent 91.84210526315789 nearest_percent 93.0 reached_pp 43.418 nearest_expected_pp 86.39376236229536 beatmapset_id 984684 beatmap_id 2060302 mod 0
PP is not correct percent 87.21311475409837 nearest_percent 85.0 reached_pp 43.2507 nearest_expected_pp 75.96636307725159 beatmapset_id 773801 beatmap_id 1626530 mod 0
PP is not correct percent 93.75 nearest_percent 93.0 reached_pp 42.3872 nearest_expected_pp 63.00906496581458 beatmapset_id 1249048 beatmap_id 2596017 mod 0
PP is not correct percent 92.6906779661017 nearest_percent 93.0 reached_pp 42.2396 nearest_expected_pp 72.75692705035009 beatmapset_id 50771 beatmap_id 155929 mod 0
PP is not correct percent 86.12303290414879 nearest_percent 85.0 reached_pp 42.1761 nearest_expected_pp 64.2753042814976 beatmapset_id 1174505 beatmap_id 2449792 mod 0
PP is not correct percent 95.6549935149157 nearest_percent 96.0 reached_pp 41.7493 nearest_expected_pp 61.99673594011707 beatmapset_id 264147 beatmap_id 606613 mod 0
PP is not correct percent 95.3125 nearest_percent 95.0 reached_pp 39.8072 nearest_expected_pp 49.88650205459918 beatmapset_id 558938 beatmap_id 1182213 mod 0
PP is not correct percent 92.02334630350194 nearest_percent 93.0 reached_pp 39.7594 nearest_expected_pp 71.89082298170204 beatmapset_id 942642 beatmap_id 1971795 mod 0
PP is not correct percent 91.86307519640853 nearest_percent 93.0 reached_pp 39.2603 nearest_expected_pp 58.57124487798674 beatmapset_id 261145 beatmap_id 597469 mod 0
MaxOhn commented 1 year ago

I don't really understand what any of your output means, checked some of your map ids and I got very accurate values.

What are you comparing rosu-pp's values with?

EnricoBaivo commented 1 year ago

Sorry, for the bad description.

I will get into detail now, i would also bet on it that i do something wrong.

Im comparing with achieved BEST USER SCORES so i am aware that the pp are slightly under the expected "full pp".

I tryed to reconstruct the score pp value with respektet weighting. So i checked for big pp diffrences from the calculations. 

def main():
    # fresh fetch from OSU API For User
    with open(file="score_calculated_pps.json", mode="r", encoding="utf-8") as f:
        scores = json.load(f)
    # calculate_pp(scores)
    for score in scores:
        test_calculate_pp(score)

def test_calculate_pp(score):
    beatmaps = Path(os.getcwd()+"/../OsuHub-DATA/%s" %
                    str(score.get("beatmapset").get("id"))).glob("*.osu")
    print("Beatmapset ID: %s" % (score.get("beatmapset").get("id")))
    for osu_file_path in beatmaps:
        # find beatmap via checksum
        md5 = generate_md5(osu_file_path)
        bmap = Beatmap(path=osu_file_path.as_posix())
        calculator = Calculator(n300=score.get("statistics").get("count_300"),
                                n100=score.get("statistics").get("count_100"),
                                n50=score.get("statistics").get("count_50"),
                                n_misses=score.get(
                                    "statistics").get("count_miss"),
                                n_geki=score.get("statistics").get(
                                    "count_geki"),
                                n_katu=score.get("statistics").get(
                                    "count_katu"),
                                mods=models.mods.Mods(score.get("mods")).bitwise)
        res = calculator.performance(bmap)
        score_pp = score.get("pp")
        if md5 == score.get("beatmap").get("checksum"):
            diffrence = round(res.pp-score_pp , 2)
            if check_if_pp_is_in_range(reached_pp=score_pp, nearest_expected_pp=res.pp):
                # if is in range between 0.8 and 1.1 of the pp
                if diffrence == 0.0:
                    print("MATCHING | Rosu PP: %s  Score PP: %s diffrence %s" %
                          (res.pp, score_pp, diffrence))

            else:
                print("NOT MATCHING | Rosu PP: %s  Score PP: %s diffrence %s" %
                      (res.pp, score_pp, diffrence))

RESULT: Beatmapset ID: 399061 MATCHING | Rosu PP: 69.63567769262768 Score PP: 69.6353 diffrence 0.0 Beatmapset ID: 770306 NOT MATCHING | Rosu PP: 94.30856476996237 Score PP: 69.4847 diffrence 24.82 Beatmapset ID: 358451 Beatmapset ID: 1280506 MATCHING | Rosu PP: 68.30147624582666 Score PP: 68.3012 diffrence 0.0 Beatmapset ID: 1525595 ....

I first encountered this problem when I calculated each beatmap for a lot of mods and accuracies. Some results were accurate, and some were not. I have to mention that I use a for loop with a multiprocessing approach for calculating each beatmap in a beatmapset concurrently. Each process gets its own Calculator().

bmap = Beatmap(path=osu_file_path)
calc = Calculator()
for mod_flag in mod_combinations:
        calc.set_mods(mod_flag)
        SS = calc.performance(bmap)
        calc.set_difficulty(SS.difficulty)
        for acc in all_percents:
                 calc.set_acc(acc_percent)
                 performance_attributes = calc.performance(bmap)

Result: WOW_PP.csv MATCHING BEATMAP: checksum "24eac3ecc5a97d3bcbcf32ecf84b14bf" https://osu.ppy.sh/scores/osu/4300840266 ROSU_PP ac:97 1128.927550598182pp Mrekk ac:97,89% 521pp

I would be happy if you could point me out the problem or if it is indeed a bug with rosu-pp-python. Thanks for youre time before hand..

MaxOhn commented 1 year ago 
calculator = Calculator(n300=score.get("statistics").get("count_300"),
                        n100=score.get("statistics").get("count_100"),
                        n50=score.get("statistics").get("count_50"),
                        n_misses=score.get(
                            "statistics").get("count_miss"),
                        n_geki=score.get("statistics").get(
                            "count_geki"),
                        n_katu=score.get("statistics").get(
                            "count_katu"),
                        mods=models.mods.Mods(score.get("mods")).bitwise)

Looks like you forgot to specify the combo.

Other than that I can't spot anything wrong so maybe that will fix it already.

EnricoBaivo commented 1 year ago

Okay, MaxOhn it was my mistake. Thank you for your time.