[S-00-2] Problem Survey

[CUN-bjy]

Survey : the way to manipulate for block-stacking

Approach from TAMP

• Classical TAMP vs Learning-based TAMP
• Classical TAMP

Approach from Hierarchical RL

• nikhilbarhate99/Hierarchical-Actor-Critic-HAC-PyTorch
• Hierarchical Deep Reinforcement Learning (HDQN)
• Learning Multi-Level Hierarchies with Hindsight(HAC)
• Introduce to HRL
• Meta-Learning shared Hierarchy(MLSH)
• Article : The Promise of Hierarchical Reinforcement Learning

Multi-step Manipulation(for long-horizon tasks)

• A Long Horizon Planning Framework for Manipulating Rigid Pointcloud Objects -> this paper just introduces to make subgoal planning by representation learning of objects pointcloud
  • its manipulation skills from here
• Hierarchical Reinforcement Learning with Universal Policies for Multi-Step Robotic Manipulation, https://www.youtube.com/watch?v=n_wQuf4r0qk
• Planning for Multi-Stage Forceful Manipulation

Multi-goal Task Approach(UMDP)

• Hindsight Experience Replay
• CURIOUS: Intrinsically Motivated Modular Multi-Goal Reinforcement Learning

Approach from Relational RL

• Towards Practical Multi-Object Manipulation using Relational Reinforcement Learning

Good Examples

• IanYangChina/pybullet_multigoal_gym, paper : https://arxiv.org/pdf/2105.05985.pdf
• qgallouedec/panda-gym -> pre-trained model on rl-baselines3-zoo

---

Other Interesting Works(Advanced Works)

• IKEA Furniture Assembly Environment for Long-Horizon Complex Manipulation Tasks, (https://clvrai.github.io/furniture)
• Learning to guide task and motion planning using score-space representation
• Self-play for goal discovery in robotics manipulation
• Transporter Network: Rearranging the Visual World for Robotic Manipulation
• CAVIN: Dynamics Learning with Cascaded Variational Inference for Multi-Step Manipulation
• Neural Task Programming: Learning to Generalize Across Hierarchical Tasks

[CUN-bjy]

위 Issue에 추가해주세요!!~ @benthebear93

[CUN-bjy]

from CURIOUS: Intrinsically Motivated Modular Multi-Goal Reinforcement Learning

[CUN-bjy]

Comparison of gym environments

• pybullet_multigoal_gym

  1. kuka-single-step-task
     • push
     • reach
     • slide
     • pick and place
  2. kuka-multi-step-task
     • BlockStack
     • BlockRearrange
     • ChestPickandPlace
     • ChestPush
  3. kuka-hierarchical-pick-and-place
     • HierarchicalPickandPlace

• panda-gym

  • pick and place
  • push
  • reach
  • slide
  • stack

[CUN-bjy]

Stage1. Single-step Task Stage2. Multi-step Task(Hierarchical e.g. UOF, HAC) Stage3. goal generator(Relational RL based or Intrinsic Reward(Curiosity))

[CUN-bjy]

Above Survey list is transferred to here

[CUN-bjy]

we can survey more using this. https://github.com/Mehooz/awesome-long-horizon-goal-reaching