[Week06] migrate to gymnasium

[laktionov]

• Support new interfaces in all files and notebooks
• Change done to terminated or truncated to iterate over env
• Change done to terminated to check whether bootstrap or not when game ends
• Update video recording code
• Replace custom to_one_hot with F.one_hot in reinforce_pytorch.ipynb
• Update CartPole to v1 in reinforce_pytorch.ipynb
• Remove redundant imports from py files
• In atariwrappers.py add SwapImageAxes wrapper to permute image axes and normalize pixels, replace TFSummaries with TensorboardSummaries to log metrics in tensorboard, add auxiliary class `thunk` (similar to this )

[review-notebook-app[bot]]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[laktionov]

Have you tested this end-to-end?

Yes, I've run my version of the solved assignment in Colab

[dniku]

Thanks!