[QUESTION] Issue with Evaluating Decision Transformer Using Evaluators in d3rlpy

[XiudingCai]

I have encountered an issue while trying to evaluate the performance of the Decision Transformer (DT) using the d3rlpy library. Unlike other methods such as CQL, it seems that DT does not support passing evaluators like:

evaluators={
    'action_diff': d3rlpy.metrics.ContinuousActionDiffEvaluator(test_episodes),
}

This limitation is problematic, especially in settings where an environment is not available for evaluation. It hinders the ability to compare the performance of DT with other methods under these conditions.

Is there a workaround or a recommended approach for this situation?

Thanks so much! :>

[takuseno]

@XiudingCai Hi, sorry for the late response. This is a little tricky issue because Q-learning and Decision Transformer are completely different algorithms and it's difficult to share some interface. One possible workaround is to use callback option at fit method. You can add arbitrary logics at every steps.

https://github.com/takuseno/d3rlpy/blob/3433de5d7c9d98c03549f8bdae5552b88d37e756/d3rlpy/algos/transformer/base.py#L391