search

google-research / rliable

[NeurIPS'21 Outstanding Paper] Library for reliable evaluation on RL and ML benchmarks, even with only a handful of seeds.
https://agarwl.github.io/rliable
Apache License 2.0
765 stars 47 forks source link

Bug in plot_utils.py #11

Closed zhefan closed 2 years ago

zhefan commented 2 years ago

Hi,

In plot_utils.py, I think this line ought to be algorithms = list(point_estimates.keys()) https://github.com/google-research/rliable/blob/72fc16c31c4021b72e7b21f3ba915e1b38cff481/rliable/plot_utils.py#L245 Otherwise, algorithms cannot be indexed in the next line.

agarwl commented 2 years ago

Yes, it seems that the other plotting functions use list(point_estimates.keys()) but missed it here. Thanks for catching this .. there are tests for main library functions but not for the plotting functions. Will fix.

zhefan commented 2 years ago

Thanks for fixing it!