Continual Reinforcement Learning with Decision Transformer

[chufanchen]

Task	L2M	LoRA
hammer-v2	Cell	Cell
push-wall-v2	Cell	Cell
faucet-close-v2	Cell	Cell
push-back-v2	Cell	Cell
stick-pull-v2	Cell	Cell
handle-press-side-v2	Cell	Cell
push-v2	Cell	Cell
shelf-place-v2	Cell	Cell
window-close-v2	Cell	Cell
peg-unplug-side-v2	Cell	Cell

[chufanchen]

Try Slow Learner #158

• different lr for lora and classifier

[chufanchen]

PCA and t-SNE

[chufanchen]

LoRA init

• constant weight: 0, 0.5
• fisher information matrix

\theta_t = (1-\alpha)\theta_{t-1}^\star + \alpha \theta_{t-2}^\star

[chufanchen]

Rehearsal

• replay buffer
• generative model(gaussian, kmeans, diffusion)

129