Reinforcement Learning, 3D Understanding, Robotics

[NirViaje]

Appetizer for 3D, Neural rendering with GAN, GIRAFFE, CVPR2021 best paper

• https://arxiv.org/pdf/2011.12100.pdf
• https://zhuanlan.zhihu.com/p/382982915
• https://zhuanlan.zhihu.com/p/384364929

Topics

• Reinforcement Learning
• 3D Data Understanding
• Robotics

• https://robotics.sciencemag.org/content/5/47
  • https://arxiv.org/pdf/1901.08652.pdf
• https://rsl.ethz.ch/robots-media/anymal.html

Memory System

• https://qbi.uq.edu.au/brain-basics/memory/types-memory
• https://zhuanlan.zhihu.com/p/59808789

• https://zhuanlan.zhihu.com/p/25050071
• https://www.zhihu.com/search?q=procedural%20memory

Overview

• https://www.mdpi.com/1424-8220/20/3/939

• https://hollygrimm.com/rl_mc

• https://zhuanlan.zhihu.com/p/26174099

• Deep Learning for Video Game Playing (a review)
  • https://arxiv.org/pdf/1708.07902.pdf

• Model-Based Deep Reinforcement Learning for High-Dimensional Problems, a Survey, LIACS, Netherlands, 2020

[NirViaje]

on Reinforcement Learning

• Rl#9: 16.04.2020 Model-Based Reinforcement Learning
• https://spinningup.qiwihui.com/zh_CN/latest/spinningup/rl_intro2.html

Courses

• Advanced Topics in Deep Reinforcement learning
  • https://deeppavlov.ai/rl_course_2020
• David Silver
  • https://www.davidsilver.uk/teaching/
• CS 285 at UC Berkeley
  • http://rail.eecs.berkeley.edu/deeprlcourse/
• https://web.stanford.edu/class/cs234/

• https://github.com/AI4Finance-LLC/ElegantRL
  • 如何选择深度强化学习算法？MuZero/SAC/PPO/TD3/DDPG/DQN/等
  • 深度强化学习调参技巧：以D3QN、TD3、PPO、SAC算法为例
  • https://zhuanlan.zhihu.com/p/127792558

• Reinforce-PG-PPO

• https://deepmind.com/blog/article/Agent57-Outperforming-the-human-Atari-benchmark
• https://qiita.com/pocokhc/items/8684c6c96d3d2963e284
  • https://github.com/pocokhc/agent57

• Re：从零开始的Multi-armed Bandit

• DreamerV1 vs SimPLe (Kaiser et al., 2019)

• https://ai.googleblog.com/2020/03/introducing-dreamer-scalable.html

the timeline -

Video predication

1. https://worldmodels.github.io/ 2018.03
2. SV2P: Stochastic Variational Video Prediction, 2018.03

Dreamer

1. https://ai.googleblog.com/2019/02/introducing-planet-deep-planning.html
2. https://ai.googleblog.com/2019/03/simulated-policy-learning-in-video.html
3. https://ai.googleblog.com/2020/03/introducing-dreamer-scalable.html
4. https://ai.googleblog.com/2021/02/mastering-atari-with-discrete-world.html

OpenAI Spinning Up

• https://spinningup.openai.com/en/latest/
  • https://spinningup.qiwihui.com/zh_CN/latest/spinningup/rl_intro2.html
• https://www.zhihu.com/search?q=spinningup.openai

Google Colab

• https://www.zhihu.com/search?q=colab
• Colab on TPU
  • https://zhuanlan.zhihu.com/p/61503222
• https://zhuanlan.zhihu.com/p/56581879

some clue

• Reward/Policy/States
• on policy/off policy
  • (behavior&target)/(behavior|target)
  • (exploitation|exploration)
  • https://www.zhihu.com/question/57159315/answer/465865135
• Q-learning
  • off-policy
  • Q: action-utility function
  • The 'Q' in Q-learning stands for quality. Quality in this case represents how useful a given action is in gaining some future reward.
• OpenAI Five

• https://zhuanlan.zhihu.com/p/68391151
• https://zhuanlan.zhihu.com/p/60441731

• https://zhuanlan.zhihu.com/p/72234324

• https://www2.eecs.berkeley.edu/Pubs/TechRpts/2016/EECS-2016-217.pdf
• https://zhuanlan.zhihu.com/p/28549596

• https://intellabs.github.io/coach/components/agents/policy_optimization/sac.html
• https://zhuanlan.zhihu.com/p/70360272

DDPG

• https://blog.csdn.net/kenneth_yu/article/details/78781901

• https://www.researchgate.net/publication/339651408_Federated_Reinforcement_Learning_for_Training_Control_Policies_on_Multiple_IoT_Devices

[NirViaje]

3D understanding

• Deep Learning for 3D Point Clouds: A Survey, nudt, 2020
  • https://github.com/QingyongHu/SoTA-Point-Cloud
• Tutorial on 3D Deep Learning, UCSD, 2019

Serveral

• PointNet/PointNet++
• DeepSDF
• PIFuHD

Any kind of rotation invariant works? what about scale?

PointNet/PointNet++

• https://web.stanford.edu/~rqi/pointnet/docs/cvpr17_pointnet_slides.pdf

DeepSDF

PIFuHD

[NirViaje]

• Samples
• Architecture
• Representation

• https://openai.com/blog/procgen-benchmark/ Connected Papers
• https://github.com/openai/train-procgen
• https://github.com/openai/procgen

State Representation Learning

State Representation Learning for Control: An Overview, 2018

• https://github.com/TLESORT/State-Representation-Learning-An-Overview
• https://arxiv.org/pdf/1802.04181.pdf

Another awesome list

• https://github.com/Mehooz/awesome-representation-learning

• Action and Perception as Divergence Minimization, DeepMind, 2020

Combinatorial Optimization

• Machine Learning for Combinatorial Optimization: a Methodological Tour d’Horizon∗
  • https://www.zhihu.com/question/43610101/answer/944394769
• ATTENTION, LEARN TO SOLVE ROUTING PROBLEMS!

Skills transfer

NEURIPS 2019 WORKSHOP

• https://www.skillsworkshop.ai/schedule.html
• David Ha
  • https://slideslive.cz/38922871/invited-talk-innate-bodies-innate-brains-and-innate-world-models
  • https://blog.otoro.net/

Task-Agnostic Reinforcement Learning, Workshop at ICLR, 06 May 2019, New Orleans

Building agents that explore and learn in the absence of rewards

• https://tarl2019.github.io/

• https://slideslive.com/38915818/generalized-value-functions-knowledge-representation-for-rl-agents

Raia Hadsell, DeepMind@GECCO

• https://slideslive.com/38918216/challenges-for-learning-in-complex-enviroments
• https://www.youtube.com/watch?v=2yjWDNXYh5s

Curriculum Learning

• https://lilianweng.github.io/lil-log/2020/01/29/curriculum-for-reinforcement-learning.html
• Curriculum Learning for Robotics初窥
  • Solving Rubik's Cube With a Robot Hand, OpenAI
  • Learning Quadrupedal Locomotion over Challenging Terrain, ETHZ
  • https://leggedrobotics.github.io/rl-blindloco/

• A Comprehensive Survey on Curriculum Learning, 2020, THU

Curiosity Driven Exploration

Curiosity-driven Exploration by Self-supervised Prediction

• https://pathak22.github.io/noreward-rl/resources/icml17.pdf
• https://zhuanlan.zhihu.com/p/31920867

Modularity, Hierarchical

Flood Sung：从Modularity到Generalization

• https://zhuanlan.zhihu.com/p/137082457
• https://zhuanlan.zhihu.com/p/135332448
• https://github.com/floodsung/Deep-Reasoning-Papers

Hierarchical

• https://arxiv.org/abs/1811.12927

• https://thegradient.pub/the-promise-of-hierarchical-reinforcement-learning/
  • https://www.infoq.cn/article/cur-hzfopk7t0jqwh0gq

Transformer for RL

• Stabilizing Transformers for Reinforcement Learning, CMU, 2019
  • https://zhuanlan.zhihu.com/p/264236502
• Transformer Based Reinforcement Learning For Games, 2019

Self-Play for Automatic Goal Discovery

• Asymmetric self-play for automatic goal discovery in robotic manipulation, OpenAI, 2021
  • https://arxiv.org/pdf/2101.04882.pdf
  • https://robotics-self-play.github.io/
  • https://www.youtube.com/watch?v=_LraDHfHKnA

Self-Supervised Learning

• BYOL与SimSiam
• 图解SimCLR框架

Meta-Learning/Oneshot to RL

Learning to (Meta) Reinforcement Learning by Imitation

• https://zhuanlan.zhihu.com/p/45845001

Sim2Real

• Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics- a Survey, Finland, 2020
• DRL for Tactile Robotics: Learning to Type on a Braille Keyboard
  • https://github.com/ac-93/braille_rl
  • https://arxiv.org/pdf/2008.02646.pdf

[lucasshen123]

University of Toronto, CSC2621 Winter 2020 : Topics in Robotics Reinforcement Learning in Robotics

• https://www.pair.toronto.edu/csc2621-w20/assets/slides/lec7_dreamtocontrol.pdf
  • https://www.pair.toronto.edu/csc2621-w20/

One-hot VAE

• Building Autoencoders on Sparse, One Hot Encoded Data by Nick Hespe Towards Data Science.pdf

  • https://towardsdatascience.com/building-autoencoders-on-sparse-one-hot-encoded-data-53eefdfdbcc7