search

danijar / crafter

Benchmarking the Spectrum of Agent Capabilities
https://danijar.com/crafter
MIT License
376 stars 64 forks source link

perhaps add IRIS, DART #26

Open wassname opened 4 months ago

wassname commented 4 months ago

See Appendix 1 in https://arxiv.org/html/2406.01361v1

Model DreamerV3 (2023 preprint) IRIS DART
Steps 1M 1M 1M
Return 11.7±1.9 9.23±0.56 12.2±1.67
wassname commented 4 months ago

Also we have delta-iris coming out, see Table 1 in https://openreview.net/forum?id=BiWIERWBFX. Although confusingly, it has an IRIS score that disagree with the above paper.

Return of 5.5 @ 1M

wassname commented 4 months ago

P.S. A new version of crafter came out, in the craftax repo 100x faster. You might want to give it an official nod if you think the craftax_classic is a suitable substitute for crafter.