search

castorini / castor

PyTorch deep learning models for text processing
http://castor.ai/
Apache License 2.0
178 stars 58 forks source link

Kim CNN: replicate results on other datasets #58

Open lintool opened 7 years ago

lintool commented 7 years ago

Replicate results on all the datasets used in the initial Kim paper, along with any other common ones...

Victor0118 commented 7 years ago

Result of Kim CNN (multichannel) on Subjectivity dataset (Pang and Lee, 2004) with 10-fold CV: 92.9%, while original paper reports 93.2%.

Victor0118 commented 7 years ago

Results of Kim CNN on SST-2 dataset

Setup rand static non-static Multichannel
Original paper 82.7 86.8 87.2 88.1
Our Kim-CNN 76.55 84.78 86.10 86.16

There is a big gap between our implementation and the original paper. My implementation is here.

It might be a hyper-parameter tuning issue. Will do it in the future.

lintool commented 7 years ago

@Impavidity can you also look at this results on SST-2 and the subjectivity dataset? Might be an issue of hyperparameter tuning?