controlller에서 환경 분리, Env 폴더 생성

[Geonhee-LEE]

controlller에서 환경 분리, Env 폴더 생성 추 후 유틸리티(plotting)도 분리 필요

• [x] #25
• [ ] pure_pursuit.py
• [ ] stanley_controller.py
• [ ] state_lattice_planner.py
• [ ] cubic_spline_planner.py
• [ ] stanley_controller.py
• [ ] lqr_steer_control.py
• [ ] lqr_speed_steer_control.py
• [ ] model_predictive_speed_and_steer_control.py
• [ ] cgmres_nmpc.py
• [ ] nmpc_collision_avoidance.py
• [ ] velocity_obstacle.py

[Geonhee-LEE]

https://www.notion.so/geonhee-lee/Mobile-Robot-Control-Classic-control-Optimal-control-Learning-based-control-bbc82dbc68ce48e593525fefa532e32d#6b048fdb16714c07abcea74764a8652e

[Geonhee-LEE]

https://github.com/Geonhee-LEE/mobile_robot_control/issues/10 와도 연관

[Geonhee-LEE]

CrowdNav/crowd_sim 에서 얘기하는 Simulation Framework 참조:

• Environment
  • The environment contains n+1 agents.
    • N of them are humans controlled by certain unknown policy.
    • The other(+1) is robot and it's controlled by one known policy.
  • The environment is built on top of OpenAI gym library, and has implemented two abstract methods.
    • reset():
      • the environment will reset positions for all the agents and return observation for robot.
      • Observation for one agent is the observable states of all other agents.
    • step(action):
      • taking action of the robot as input, the environment computes observation for each agent and call agent.act(observation) to get actions of agents.
      • Then environment detects whether there is a collision between agents.
        • If not, the states of agents will be updated.
      • Then observation, reward, done will be returned.

• Agent

  • Agent is a base class, and has two derived class of human and robot.

    • Agent class holds all the physical properties of an agent, including position, velocity, orientation, policy and etc.

      • visibility: humans are always visible, but robot can be set to be visible or invisible

      • sensor: can be either visual input or coordinate input

        • [CrowdNav]

          # compute the observation
          if self.robot.sensor == 'coordinates':
              ob = [human.get_observable_state() for human in self.humans]
          elif self.robot.sensor == 'RGB':
              raise NotImplementedError

      • kinematics: can be either holonomic (move in any direction) or unicycle (has rotation constraints)

        def step(self, action, update=True):
            ...
            if self.robot.kinematics == 'holonomic':
                vx = human.vx - action.vx
                vy = human.vy - action.vy
            else:
                vx = human.vx - action.v * np.cos(action.r + self.robot.theta)
                vy = human.vy - action.v * np.sin(action.r + self.robot.theta)

      • act(observation): transform observation to state and pass it to policy

  • CrowdNav/crowd_sim
    • class Agent(object):
      • def init(self, config, section):
      • def print_info(self):
      • def set_policy(self, policy):
      • def sample_random_attributes(self):
      • def set(self, px, py, gx, gy, vx, vy, theta, radius=None, v_pref=None):
      • def get_observable_state(self):
      • def get_next_observable_state(self, action):
      • def get_full_state(self):
      • def get_position(self):
      • def set_position(self, position):
      • def get_goal_position(self):
      • def get_velocity(self):
      • def set_velocity(self, velocity):
      • @abc.abstractmethod def act(self, ob): # Compute state using received observation and pass it to policy
      • def check_validity(self, action):
      • def compute_position(self, action, delta_t):
      • def step(self, action): # Perform an action and update the state
      • def reached_destination(self):
  • [CrowdNav](https://github.com/vita-epfl/CrowdNav) # Agent 상속
    • class Robot(Agent):
      • def init(self, config, section):
      • def act(self, ob):
  • CrowdNav/crowd_sim/envs/utils/human.py** # Agent 상속
    • class Human(Agent):
      • def init(self, config, section):
      • def act(self, ob): # The state for human is its full state and all other agents' observable states
• Policy
  • Policy takes state as input and output an action.
  • Current available policies:
    • ORCA: compute collision-free velocity under the reciprocal assumption
    • CADRL: learn a value network to predict the value of a state and during inference it predicts action for the most important human
    • LSTM-RL: use lstm to encode the human states into one fixed-length vector
    • SARL: use pairwise interaction module to model human-robot interaction and use self-attention to aggregate humans' information
    • OM-SARL: extend SARL by encoding intra-human interaction with a local map
• State
  • There are multiple definition of states in different cases.
  • The state of an agent representing all the knowledge of environments is defined as JointState, and it's different from the state of the whole environment.
    • ObservableState: position, velocity, radius of one agent
    • FullState: position, velocity, radius, goal position, preferred velocity, rotation
    • DualState: concatenation of one agent's full state and one another agent's observable state
    • JoinState: concatenation of one agent's full state and all other agents' observable states
• Action

[Geonhee-LEE]

python run.py:

https://user-images.githubusercontent.com/12562147/159394715-de3de3cf-cf98-4baa-b83c-5ca19e877aaf.mp4