Closed vaidas-sl closed 10 months ago
I'm letting Lucas review this since he's more knowledgeable about this algorithm. Makes sense to me but surely impacts performance.
Hi @vaidas-sl, thanks for the PR!
Although in the original PCN code (https://github.com/mathieu-reymond/pareto-conditioned-networks/blob/main/pcn/pcn.py#L186C69-L186C69) the evaluation during training uses the stochastic policy, they evaluate using a deterministic policy after training.
I think it makes sense that the evaluation results that we report are obtained using the deterministic policy, as it is the policy that matters at the end of training.
According to PCN paper, "at execution time — i.e., after the training process — we use a deterministic policy by systematically selecting the action with the highest confidence" however current implementation always uses sampled actions