search

foxlf823 / Multi-Filter-Residual-Convolutional-Neural-Network

Multi-Filter Residual Convolutional Neural Network for Text Classification
45 stars 15 forks source link

Cannot reproduce the reported results #11

Open xdwang0726 opened 2 years ago

xdwang0726 commented 2 years ago

Hi,

Thanks for the great work! I am trying to reproduce the reported results, however, I followed the exactly steps but got 

'prec_at_8': 0.6807161921708185

which is far lower comparing to the reported results 0.734. I am wondering is there any hidden tricks that are not reported in the paper? Thanks!

foxlf823 commented 2 years ago

I think the results can be reproduced without much effort. Usually, there may be several points that should be paid attention (1) Make sure the process of preparing data is correct. (2) You use the same pretrained word embeddings. (3) hyper-parameters If you still can't reproduce after checking these points, you can send your logging files to foxlf823@gmail.com so that I can help analyze.

xdwang0726 commented 2 years ago

Thanks @foxlf823 I could reproduce the results now.

One more question, in inference(see line below), you use the loss function BCEWithLogitsLoss, and then again add sigmoid to the output. Is this process duplicated (like did sigmoid twice)? https://github.com/foxlf823/Multi-Filter-Residual-Convolutional-Neural-Network/blob/4acb3c0b68a48b125f042a066b5eb60019c07de5/train_test.py#L89