Diversity is All You Need: Learning Skills without a Reward Function

[ven-kyoshiro]

Paper

• https://arxiv.org/abs/1802.06070

  Branch

• diayn

[ven-kyoshiro]

・DIAYN on notebook

• [x] prepare HalfCheetah of pybullet

• [x] search about half cheater preprocessing

  • not need

• [ ] learn HalfCheetah with SAC

  • hard for laptop
  • use server
  • I heard it takes 2days to learn HalfCheetah so change the task to ReacherBulletEnv-v0

• [x] learn Reacher-v0 with SAC

  • bullet default reward "log_dir = sac_bullet_reacher_defo_reward"
  • ReacherBulletEnv-v0's reward setting is differ from mujoco.
  • I changed this setting to like a mujoco. pybullet_envs/gym_manipulator_envs.py

    l30 self.rewards = [float(self.potential - potential_old), float(electricity_cost), float(stuck_joint_cost)]

  to

  l30 self.rewards = [float(self.potential), float(electricity_cost), float(stuck_joint_cost)]

• [x] define discriminator

• [x] incorporate discriminator

  • training
  • use as reward

• [x] make evaluation code

  • RL epoch vs Discriminator epoch
  • observation of reacher in pybullet
    • target_x,
    • target_y,
    • self.to_target_vec[0],
    • self.to_target_vec[1],
    • np.cos(theta),
    • np.sin(theta),
    • self.theta_dot,
    • self.gamma,
    • self.gamma_dot,

• [x] DIAYN on Reacher

• [x] save discriminator_loss

• [x] implements to the function to save halfway models

• [x] discriminator didn't work on GPU .

• [x] Combine default reward function and diayn

future work?

• [x] add discriminator_loss to logger - [ ] use same parameter as the paper?
• [x] tune to Reacher settings

[rarilurelo]

170