search

Aaditya-Prasad / consistency-policy

[RSS 2024] Consistency Policy: Accelerated Visuomotor Policies via Consistency Distillation
https://consistency-policy.github.io/
MIT License
87 stars 6 forks source link

Parameter of CM model setting issues for plug-in tasks in the real world #3

Closed yolo01826 closed 1 week ago

yolo01826 commented 1 month ago

Thank you very much for your work. Previously, under your guidance, we successfully completed the plug-in task using the original DP. Following the same parameter configuration from your consistency policy in the simulation task, we trained the EDM and CTMP strategies for the plug-in task. However, the resulting CTMP curve appears quite strange. Could you advise if there are any changes needed in the parameter settings for strategies in the real environment? EDM training curve as shown in the attached image. image

CTMP training curve as shown in the attached image. image image image

Thank you!

yolo01826 commented 1 month ago

@kevin-thankyou-lin @Aaditya-Prasad

Aaditya-Prasad commented 1 month ago

I have had runs with loss 'curves' that look like what you're seeing; I copied loss and mse error from one of our real world training runs below: image

We have had runs (in sim) without the loss spikes results (though you can see overfitting): image

The loss spikes do appear only in the DSM loss (1), the CTM loss (2) is much more regular: image

image

so the first thing I'd try would be to log the sampled noise scales for the DSM loss and see if the spikes happen when very ood noise scales are sampled. I didn't get a chance to try this due to timelines. It'd also be reasonable to play around with the learning rate, relative loss balancing, and making sure the dataset is properly shuffled.