search

andrew-j-levy / Hierarchical-Actor-Critc-HAC-

This repository contains the code to implement the Hierarchical Actor-Critic (HAC) algorithm.
MIT License
253 stars 61 forks source link

Clarification question regarding project_state_to_end_goal/subgoal #11

Closed ChengshuLi closed 5 years ago

ChengshuLi commented 5 years ago

Hi, I have a quick clarification question regarding project_state_to_end_goal/subgoal.

Currently they are defined to be functions of arguments sim and state.

In https://github.com/andrew-j-levy/Hierarchical-Actor-Critc-HAC-/blob/8bd543f36a9fee8ea396c267b5236555f365e28e/example_designs/PENDULUM_LAY_1_design_agent_and_env.py#L79 joint velocities are extracted from state.

Whereas in https://github.com/andrew-j-levy/Hierarchical-Actor-Critc-HAC-/blob/8bd543f36a9fee8ea396c267b5236555f365e28e/example_designs/UR5_LAY_1_design_agent_and_env.py#L88 the joint velocities are extracted from sim.

I wonder whether this is an intentional design choice and whether project_state_to_end_goal/subgoal can just be function of state. Why do we need to pass sim into the function?

Thanks a lot! I appreciate your help.

andrew-j-levy commented 5 years ago

Hello,

Yes the projection function can just be a function of the state. I included the sim object as well to give the user extra flexibility in case the user needs some metric that is not in the state space.

ChengshuLi commented 5 years ago

Got it. Thanks!