search

snumprlab / cl-alfred

Official Implementation of CL-ALFRED (ICLR'24)
https://bhkim94.github.io/projects/CL-ALFRED/
GNU General Public License v3.0
12 stars 2 forks source link

Results under different seeds #3

Closed JACK-Chen-2019 closed 3 months ago

JACK-Chen-2019 commented 3 months ago

Could you please provide the results obtained with different seeds? Thanks very much.

bhkim94 commented 3 months ago

Hi @JACK-Chen-2019,

Thank you for having an interest in our work! Here are the results of our CAMA with different seeds (for valid_unseen).

---------------------------------------------------------------------------------------------------
                          Behavior-IL                                 Environment-IL
---------------------------------------------------------------------------------------------------
 seed  |   SR_last    GC_last    SR_avg     GC_avg   |   SR_last    GC_last    SR_avg     GC_avg
---------------------------------------------------------------------------------------------------
   1   |    12.18      26.79       9.95      26.70   |    14.37      29.52      15.66      34.60
   2   |    11.94      27.03      14.84      25.29   |    15.52      30.41      18.58      38.05
   3   |    13.64      28.49      15.76      28.60   |    15.52      33.72      14.75      33.65
   4   |    13.28      29.58      12.45      24.00   |    14.37      31.42      15.18      30.03
   5   |    17.17      31.84      17.96      31.92   |    13.22      29.90      14.18      30.69
---------------------------------------------------------------------------------------------------
  avg  | 13.64±0.94 28.75±0.92 14.19±1.38 27.30±1.38 | 14.60±0.43 30.99±0.75 15.67±0.77 33.40±1.45

Feel free to reopen this if you have any further questions.

JACK-Chen-2019 commented 3 months ago

Thank you very much for your reply. Are there experimental results on validseen？

bhkim94 commented 3 months ago

No, it's from valid unseen.