liuzuxin / OSRL

🤖 Elegant implementations of offline safe RL algorithms in PyTorch
https://offline-saferl.org
Apache License 2.0
178 stars 12 forks source link

DT-Cost file missing in examples #14

Closed shahaamirbader closed 1 year ago

shahaamirbader commented 1 year ago

Hi,

I see that one of your baseline method is a simple DT-Cost. But I am unable to find the files in examples folder for DT-Cost. Can you kindly share those files for training.

Thanks

liuzuxin commented 1 year ago

You can simply enable DT-Cost mode by using our CDT implementation and set the following parameters:

# disable data augmentation
augment_percent = 0
# disable stochastic policy head
stochastic = False
shahaamirbader commented 1 year ago

Perfect, thanks. Can you explain how this will make it a simple DT+Cost based transformer.

liuzuxin commented 1 year ago

You are welcome. By removing data augmentation, the data side has no much additional operations, and by removing the stochastic policy head, the model uses the same deterministic head as DT (also without entropy regularization), so it is essentially just performing DT with an additional target cost token.

shahaamirbader commented 1 year ago

perfect thanks. This helps.