search

GriffinLiang / vrd-dsr

Code for Visual Relationship Detection with Deep Structural Ranking (AAAI2018)
122 stars 32 forks source link

Question about the experiments result #31

Open lileilai opened 5 years ago

lileilai commented 5 years ago

hi~ i have re-implement your code on vrd with the default parameters , and getting a result。The result does not accord with the paper, and the result as follows:

Epoch Pre R@50 ZS R@100 ZS Rel R@50 ZS R@100 ZS

  0     77.8957  57.4486  88.653   76.7123    15.6669   4.90534  19.4686   8.0895
  1     77.3978  57.2774  89.0068  76.113     10.4446   5.85198  14.0226   8.43373
  2     77.3978  58.7329  89.1116  76.9692     7.22178  6.36833  10.6288   8.26162
  3     75.4193  58.3904  88.0372  76.4555     6.03788  6.45439   8.98448  8.43373
  4     74.8035  59.161   87.9586  76.1986     5.41963  6.19621   8.43199  8.0895
  5     74.5283  57.6199  87.8407  76.1986     5.51171  5.76592   8.14259  8.17556
  6     74.5676  57.6199  87.8407  76.4555     5.45909  5.93804   8.18206  8.26162
  7     72.2615  58.3048  86.3732  76.1986     5.196    5.85198   7.72165  8.26162
  8     71.3574  56.9349  85.9801  75          5.02499  5.5938    7.78742  8.34768
  9     71.6195  57.5342  85.9801  75.6849     4.74875  5.5938    7.51118  8.34768

At the same time , when i cancel the note of reading the file "parms_em.pkl" , and initializing the "self.emb"， the performance declined instead.

I hope you can answer these questions.

thanks.

GriffinLiang commented 5 years ago

The performance decreases with the training time. I could not figure out what happened only from the above information. I think You should pay attention to the pytorch version (0.2.0) [torch-0.2.0.post3] and the value of the learning rate.

lileilai commented 5 years ago

Thanks for your reply~ the value of learning rate is the default 0.00001, and i also confuse the ability about the paramter of "loca_type" that is setted zero in default , and does this parm affect the model's performance ?

lileilai commented 5 years ago

My code runs on the environment of pytorch 0.4 , i will change to 0.2 . Thanks for your reply.

JLHins commented 5 years ago

@lileilai can you get the same results as the paper shows? I have met the same problem as yours

YuxuanHe04 commented 3 years ago

@lileilai Hello, have you solved this problem by changing pytorch to 0.2? I have met the same problems as yours.