xiph / awcy

http://arewecompressedyet.com/
MIT License
70 stars 46 forks source link

bd_rate_report.py cannot compare runs with different number of quality levels #305

Open KyleSiefring opened 2 years ago

KyleSiefring commented 2 years ago

From json returned by awcy


  mk = (y[1:] - y[:-1]) / hk
./bd_rate_report.py:105: RuntimeWarning: invalid value encountered in true_divide
  whmean = (w1/mk[:-1] + w2/mk[1:]) / (w1 + w2)
./bd_rate_report.py:64: RuntimeWarning: invalid value encountered in multiply
  d = ((2*h0 + h1)*m0 - h0*m1) / (h0 + h1)
./bd_rate_report.py:316: RuntimeWarning: invalid value encountered in true_divide
  decode_times = (decode_times_b - decode_times_a) / decode_times_a
Traceback (most recent call last):
  File "./bd_rate_report.py", line 356, in <module>
    metric_data[video] = bdrate(args.run[0]+'/'+task+'/'+video+args.suffix,args.run[1]+'/'+task+'/'+video+args.suffix,None,args.fullrange)
  File "./bd_rate_report.py", line 308, in bdrate
    encode_times = (encode_times_b - encode_times_a) / encode_times_a
ValueError: operands could not be broadcast together with shapes (4,) (5,) ```
KyleSiefring commented 2 years ago

https://beta.arewecompressedyet.com/?job=master-s3-9417a4df1069cc06%402021-10-28T21%3A18%3A58.839Z&job=lru_skip3-s1%402022-01-08T15%3A02%3A11.691Z

lru_skip3-s1 is missing Netflix_Boat_1920x1080_60fps_8bit_420_60f.y4m for whatever reason and this seems to be the secondary cause of the error.

https://beta.arewecompressedyet.com/?job=kff1-arnr-str1-fix%402022-01-31T23%3A06%3A02.691Z&job=svt-av1_0.8.8_rc1_6qp-p8%402021-12-02T12%3A21%3A09.619Z

Also doesn't work, the primary reason seems to be that awcy can't compare runs with differing numbers of quality levels.

(The division seems to be just a warning and probably doesn't matter)