ropiens / project-sandwich-man

A project for researching a complex and long-horizon manipulation task especially focused on hierarchically stacking blocks.

MIT License

5 stars 0 forks source link

[S-00-1] Project Planning #1

Closed CUN-bjy closed 3 years ago

CUN-bjy commented 3 years ago

Let's start the project-sandwich-man kick-off : 21/08/04

CUN-bjy commented 3 years ago

Survey

Approach from TAMP

Approach from Hierarchical RL

Good Examples

https://github.com/IanYangChina/pybullet_multigoal_gym

CUN-bjy commented 3 years ago

~8.20 까지 Survey 기간

CUN-bjy commented 3 years ago

Goal

stacking problem

[ ] multi-stage task 수행
[ ] multi-task(pick and place) 수행
[ ] multi-goal task 수행
[ ] (옵션) task를 주는 방식
- (1) description (빨강, 파랑, 노랑) -> 순서로 쌓기
- (2) 이미지를 통해 전달(object detection x) --> deep visual reasoning
[ ] (옵션) object위치, 색깔 랜덤

(meta-learning??)

problem survey -> #2

CUN-bjy commented 3 years ago

Stage1. Single-step Task Stage2. UOF, HAC, Multi-step Task(Hierarchical) Stage3. Relational RL based -> goal shaping(Intrinsic Reward(Curiosity))

CUN-bjy commented 3 years ago

Development Plan

Survey(#2)
Single-step Stacking
- Multi-goal problem -> sparse reward and hindsight concept
  - (DDPG+HER, SAC+HER, TQC+HER, etc.)
- experiments for various environment
Multi-step Stacking
- Hierarchical Structure
  - Option Framework(high-level/low-level policy) (UOF etc.)
  - Multi-level Sub-goal(HAC etc.)
Goal Representation Embedding
- GAT(Graph ATtention network)-based relational goal representation
- Intrinsic Motivated Goal Modular(CURIOUS etc.)
Further More
- Vision-based Goal Embedding
- Curiosity-based Goal Discovery

CUN-bjy commented 3 years ago

@benthebear93 See the plan and feedback pls.