search

takuseno / d3rlpy

An offline deep reinforcement learning library
https://takuseno.github.io/d3rlpy
MIT License
1.29k stars 230 forks source link

[QUESTION] Using n_frames for Tabular data #262

Open paulhilders opened 1 year ago

paulhilders commented 1 year ago

Hi! The documentation states that the "n_frames" argument for the algorithms is used to enable frame stacking for image observation. I was wondering if the use of n_frames can directly be applied for tabular (vector) data, so that vectors may be stacked, or if n_frames should only be used for image data.

The purpose for using stacking with the vector data would be to incorporate part of the most recent history into the input for the model.

Kind regards.

takuseno commented 1 year ago

@paulhilders Hi, currently, this is not supported yet. I'm working on the next major update. In the next version, you can do it with vector arrays. https://github.com/takuseno/d3rlpy/blob/93831ba38dfad2b72aac7608fe80f2f84ec01d6e/d3rlpy/dataset/transition_pickers.py#L48

paulhilders commented 1 year ago

@takuseno That sounds great! Do you maybe have an ETA for this update? :)