search

uber-research / poet

Paired Open-Ended Trailblazer (POET) and Enhanced POET
https://eng.uber.com/poet-open-ended-deep-learning/
Apache License 2.0
240 stars 53 forks source link

ANNECS measure confusion #7

Open atalnarayan opened 2 years ago

atalnarayan commented 2 years ago

Hi,

Can you please explain the reasoning behind including archived optimizers to measure the ANNECS score? The cited line of code implies that even if an environment passes the MC for the active agents, it is not counted in ANNECS unless it passes MC for the archived agents and that too without finetuning.

robertmash2 commented 2 years ago

I too am running into similar confusion. After carefully reviewing the code, I think that it is consistent with section 3.1 of the ePOET paper and section 3 paragraph 4 of the POET paper. However, I don't see how figure 8 was generated by ANNECS measures generated by this code.  Using this code, the ANNECS measure should be zero for however many POET iterations are necessary to increase the population to the maximum number of environments, forcing the oldest into the archive. At which point the ANNECS measure can be incremented. But the ANNECS measures in the figure appear to start increasing with the first POET iterations.