[Question] The requirement for providing environment in CreateOPEInput

[XiudingCai]

I'd like to inquire whether it is necessary to provide an environment when using this library for OPE of offline RL. As code followed,

create input for OPE class

prep = CreateOPEInput( env=env, ) In some cases, we can only provide an offline collected dataset and may not be able to offer a real-time interactive environment. I would appreciate clarification on whether providing an environment is a prerequisite for utilizing CreateOPEInput for OPE evaluation in such scenarios. Thank you!

[aiueola]

Hi @XiudingCai,

Thank you for the question. The default value of env is set to None (please also refer to API reference), and CreateOPEInput also works with real-world data without gym environments.

In this case, the online evaluation is not available, and thus on_policy_policy_value in the returned input_dict will be None.

This page also provides some guidelines for using real-world datasets, and I hope this will be helpful for you.

[XiudingCai]

Thanks a lot for the reply! I am gonna try it. : >