fsktom / rusty-endsong-parser

Better, more performant version of https://github.com/fsktom/endsong-parser-python written in Rust
0 stars 0 forks source link

Better relative_to_album and relative_to_artist plots #49

Closed fsktom closed 1 year ago

fsktom commented 1 year ago

More granular

https://github.com/fsktom/rusty-endsong-parser/blob/a681a58ef775cdc2fdcd34daec77f6d64919d56a/src/plot/relative.rs#L44-L46

e.g. instead of checking the amount of plays for the album for each song listen, check the amount of song plays for each album play should result in no steep graphs

fsktom commented 1 year ago

maybe also relative (to all) that it checks till the most recent entry

performance impact for sure dunno how heavy

maybe it should be an option? idk

fsktom commented 1 year ago

trace::relative::to_artist and trace::relative::to_album have now got perfect resolution - the former goes through each artist occurrence and the latter through each album occurrence.

trace::relative::to_all goes through each aspect occurrence, but also checks the percentage an additional time for each day :D

fsktom commented 1 year ago

Yooooo I don't actually need to do this

let end = match artist_dates.binary_search(date) {
        Ok(i) => i,
        Err(i) => i - 1,
    };
let sum_of_artist_plays = artist_dates[..=end].len() as f64;

if I'm iterating over artist_dates anyways.... I can just count up.... like I did before....

Same with relative:to_all I could technically go through each entry and count up too with perfect resolution (I wonder if it's gonna be faster than what I'm currently doing which is using binary search for two indexes and then taking the length of that slice..., but it's going over the merged list of aspect plays and a date each day)

fsktom commented 1 year ago

Previous performance (on 3 endsong files)

absolute                time:   [453.75 µs 454.17 µs 454.94 µs]
Found 12 outliers among 100 measurements (12.00%)
  3 (3.00%) high mild
  9 (9.00%) high severe

relative                time:   [1.0344 ms 1.0402 ms 1.0479 ms]
Found 12 outliers among 100 measurements (12.00%)
  1 (1.00%) high mild
  11 (11.00%) high severe

relative_to_artist      time:   [976.90 µs 981.84 µs 987.92 µs]
Found 11 outliers among 100 measurements (11.00%)
  4 (4.00%) high mild
  7 (7.00%) high severe

relative_to_album       time:   [306.71 µs 308.59 µs 311.37 µs]
Found 10 outliers among 100 measurements (10.00%)
  2 (2.00%) high mild
  8 (8.00%) high severe

current:

absolute                time:   [459.04 µs 460.28 µs 461.88 µs]
                        change: [-2.5439% +0.2370% +2.0740%] (p = 0.88 > 0.05)
                        No change in performance detected.
Found 7 outliers among 100 measurements (7.00%)
  4 (4.00%) high mild
  3 (3.00%) high severe

relative                time:   [9.0060 ms 9.0304 ms 9.0687 ms]
                        change: [+759.80% +766.71% +772.59%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 16 outliers among 100 measurements (16.00%)
  7 (7.00%) high mild
  9 (9.00%) high severe

relative_to_artist      time:   [865.58 µs 865.99 µs 866.48 µs]
                        change: [-12.601% -11.895% -11.317%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 10 outliers among 100 measurements (10.00%)
  4 (4.00%) high mild
  6 (6.00%) high severe

relative_to_album       time:   [214.82 µs 214.89 µs 214.97 µs]
                        change: [-31.202% -30.710% -30.277%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
  5 (5.00%) high mild
  4 (4.00%) high severe

absolute not changing is good relative to artist and album improving is what I wanted

relative to all getting worse is expected, since its resolution now covers all entries, but I'm surprised it's that much worse...

fsktom commented 1 year ago

The Sabaton relative to all plot (.html file) ran on endsong files 0-9 now is 5.2M big with 187k lines of HTML XD

(after removing duplicate timestamps and running 0.3/10s filter)

fsktom commented 1 year ago

I think the performance is alright, so I'm gonna leave it like that :)