search

facebookresearch / vip

Official repository for "VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Training"
https://sites.google.com/view/vip-rl
Other
135 stars 17 forks source link

Visualizing 2d embeddings for VIP and TCN #3

Closed priyankamandikal closed 1 year ago

priyankamandikal commented 1 year ago

Hello, I tried reproducing the 2d embedding plot in Fig. 2 of your paper by training a VIP and TCN model on 100 demos from Franka Kitchen sdoor_open task. This is the plot that I'm getting after training a ResNet-34 model (with hidden_dim=2) for 2k iters as suggested in your paper. It is trained only with the VIP or TCN losses without any LP losses.

As you can see, the results are completely opposite to what's shown in the Fig. 2 of the VIP paper. I verified this with different runs and get the same result.

After 10k iters, I get something like this, where VIP has improved from what it was at 2k, and TCN also looks good.

I am using the TCN loss directly from the R3M repo without any changes. Do you know why the plots I'm getting do not match up with the VIP paper? Did you use any different settings for generating these plots? Thanks!

priyankamandikal commented 1 year ago

Hi Jason, just following up on this. Did you use gamma = 0.98 or gamma = 1.0 in the value function for training VIP?

JasonMa2016 commented 1 year ago

Hi Priyanka,

Thank you for your interest in VIP! We used gamma=0.98. I am not sure why the VIP curves are not as smooth in your case -- have you figured out the issue? I can try reproducing the curves if needed. In this toy case, we also set num_negatives to be 0.

priyankamandikal commented 1 year ago

Hi Jason, thanks for getting back. Yes, these plots are produced with the same settings as in https://github.com/facebookresearch/vip/blob/main/vip/cfgs/config_vip.yaml If I evaluate with snapshot_2000.pt, I get these plots, they look like the paper plots with snapshot_10000.pt though