How to train a trotting gait?

[Kobaya627]

Thanks for your wonderful repo, it works relatively good with training less than 20 minutes, however I found my trained policy (for unitree robot a1) quite unnatural, it tries to lift both FL and FR legs at the same time with very low commanding velocity, that weird behavior drives me to add some reward function to encourage trotting gait (or any other desired gaits). Actually I only found reward below that have some influence on lifting legs.

    def _reward_feet_air_time(self):
        # Reward long steps
        # Need to filter the contacts because the contact reporting of PhysX is unreliable on meshes
        contact = self.contact_forces[:, self.feet_indices, 2] > 1.
        contact_filt = torch.logical_or(contact, self.last_contacts) 
        self.last_contacts = contact
        first_contact = (self.feet_air_time > 0.) * contact_filt
        self.feet_air_time += self.dt
        rew_airTime = torch.sum((self.feet_air_time - 0.5) * first_contact, dim=1) # reward only on first contact with the ground
        rew_airTime *= torch.norm(self.commands[:, :2], dim=1) > 0.1 #no reward for zero command
        self.feet_air_time *= ~contact_filt
        return rew_airTime

It accumulates feet swing time and rewards that when first contact, it seems this reward encourage lifting legs but has nothing to do with gaits. Is there any interface that can control gait that I didn't notice ? If not, can you guys offer me some advise on how to train a trotting gait policy ?