search

xjgaocs / Trans-SVNet

26 stars 5 forks source link

Provided Transformer model gets bad results #2

Open liyunfan13 opened 2 years ago

liyunfan13 commented 2 years ago

Hi, I'm trying to reproduce the results in your paper on the Cholec80 dataset, but the provided model seems to get pretty bad results when running inference with it (around 70% acc. on my machine). I tried training the model myself, and got better results (around 85% acc.), but still short of the results in the paper. Is there anything that I might be doing wrong in the process? Thanks.

xjgaocs commented 2 years ago

We used the same data pre-processing and evlautation modules of TMRNet.

liyunfan13 commented 2 years ago

Yes, I followed their data preparation method, and used 40 videos for training, 8 for validation, 32 for testing.

xjgaocs commented 2 years ago

As written in the Trans-SVNet paper, 40/40 are used for training and testing. The evaluation used Matlab code of TMRNet.

liyunfan13 commented 2 years ago

Thanks. I will try that.

liyunfan13 commented 2 years ago

Hi, I have one more question: how did you generate the +/- results (e.g. 90.3 +/- 7.1) in the paper? Was the first number averaged over multiple runs? Thanks.