search

basaldella / deepkeyphraseextraction

Deep Networks for Keyphrase Extraction
Apache License 2.0
7 stars 4 forks source link

Inspec dataset evaluation #1

Open laurita opened 5 years ago

laurita commented 5 years ago

Hey, I see in the code that for evaluating INSPEC dataset, only uncontrolled keyphrases are used, but in this comment it says ".contr and .uncontr". What is the source of truth for achieving 0.422 F1@10 score that is mentioned in the Experimental Results section of the paper Bidirectional LSTM Recurrent Neural Network for Keyphrase Extraction? Thanks in advance! Laura

basaldella commented 5 years ago

Hello,

that paper is based on a different codebase. Anyway, the comment is a mistake; as you can see a few lines below, only .uncontr files are loaded.

Marco

laurita commented 5 years ago

Thanks for a quick reply! Could you give some more info about how the 0.422 was achieved? It seems quite a high number compared to other state-of-the art methods. Also, the paper that improves BLSTM by adding attention mechanism reports lower numbers, namely 0.394 F1@10. Is there a difference how these numbers are computed?