anuzzolese / oke-challenge-2016

22 stars 8 forks source link

About the evaluation dataseet 2016 #15

Open chabmed opened 8 years ago

chabmed commented 8 years ago

Hi, i was checking the evaluation dataset 2016 in details, but i saw that every "He" is pointing to http://dbpedia.org/resource/William_Cullen not like in the training dataset there is "he_8", "he_9" And not every personal pronouns is annotated. Thanks, Mohamed Chabchoub

rtroncy commented 8 years ago

See also this pull request where we have tried to fix numerous cases like this.

anuzzolese commented 8 years ago

I have fixed this issue in the last push. @rtroncy this fix resolves the issues reported by @jplu (with exclusion of those about the URIs). Can you please have a look at the new dataset?

jplu commented 8 years ago

Thanks @anuzzolese for the merge!

@chabmed let us know if you have any other issue.

chabmed commented 8 years ago

Hi, yes i think there is some others about the offsets, i don't know yet if there is others, but i detected the followings Wchich is between the parenthesis is the actual version of the GoldStandard ----In 1926, at the age of 24, he applied for a professorship at the Sapienza University of Rome. 27 :: he :: NIL :: Person (13 :: he :: NIL :: Person) ----Marvin Lee Minsky was born to an eye surgeon father, Henry, and to a Jewish mother, Fannie, who was an activist in Zionist affairs, in New York City, where he attended the Ethical Culture Fieldston School and the Bronx High School of Science. 156 :: he :: NIL :: Person ----Hendler held a longstanding position as professor at the University of Maryland where he was the Director of the Joint Institute for Knowledge Discovery and held joint appointments in the Department of Computer Science, the Institute for Advanced Computer Studies and the Institute for Systems Research. 86 :: he :: James_Hendler :: Person (81 :: he :: James_Hendler :: Person) ----Brown was educated first at Kirkcaldy West Primary School where he was selected for an experimental fast stream education programme, which took him two years early to Kirkcaldy High School for an academic hothouse education taught in separate classes. 64 :: he :: Gordon_Brown :: Person (59 :: he :: Gordon_Brown :: Person) ----In October 1850, already an accomplished mathematician, Maxwell left Scotland for the University of Cambridge. He initially attended Peterhouse, but before the end of his first term transferred to Trinity, where he believed it would be easier to obtain a fellowship. 212 :: he :: James_Clerk_Maxwell :: Person (207 :: he :: James_Clerk_Maxwell :: Person) ----He graduated from high school at 16, then moved to the University of Turin. Despite a strong interest for mathematics and physics, he decided to study medicine. 131 :: he :: NIL :: Person (109 :: he :: NIL :: Person) ----He attended grammar school in Belfast from the age of 12 and entered the University of Glasgow in 1746 when he was eighteen years old, studying there for four years before spending another four at the University of Edinburgh, furthering his medical studies. 108 :: he :: NIL :: Person (104 :: he :: NIL :: Person) I'll try to automatically verify each mention if it has the right offset

jplu commented 8 years ago

Cool! thanks @chabmed

@anuzzolese I will make another PR today with those fixes.

anuzzolese commented 8 years ago

Thanks @jplu

jplu commented 8 years ago

Here the corresponding PR https://github.com/anuzzolese/oke-challenge-2016/pull/18