CoreNLP's quote attribution is barely usable/has very low accuracy

The quote extraction has a pretty low accuracy, either I did something wrong or this should be stated as experimental in the docs. CoreNLP struggles with sentences with multiples speakers, with non-typical expression words, e.g. she sighed... instead of she said... and is frequently wrong with the canonical speaker.

The original text with some really hard parts is below. CoreNLP unfortunately even fails on the easy/unambiguous parts (below the og text is the result with errors being marked!):

"Christmas won't be Christmas without any presents," grumbled Jo, lying on the rug.

"It's so dreadful to be poor!" sighed Meg, looking down at her old dress.

"I don't think it's fair for some girls to have plenty of pretty things, and other girls nothing at all," added little Amy, with an injured sniff.

"We've got Father and Mother, and each other," said Beth contentedly from her corner.

The four young faces on which the firelight shone brightened at the cheerful words, but darkened again as Jo said sadly, "We haven't got Father, and shall not have him for a long time." She didn't say "perhaps never," but each silently added it, thinking of Father far away, where the fighting was.

Nobody spoke for a minute; then Meg said in an altered tone, "You know the reason Mother proposed not having any presents this Christmas was because it is going to be a hard winter for everyone; and she thinks we ought not to spend money for pleasure, when our men are suffering so in the army. We can't do much, but we can make our little sacrifices, and ought to do it gladly. But I am afraid I don't," and Meg shook her head, as she thought regretfully of all the pretty things she wanted.

"But I don't think the little we should spend would do any good. We've each got a dollar, and the army wouldn't be much helped by our giving that. I agree not to expect anything from Mother or you, but I do want to buy Undine and Sintran for myself. I've wanted it so long," said Jo, who was a bookworm.

"I planned to spend mine in new music," said Beth, with a little sigh, which no one heard but the hearth brush and kettle-holder.

"I shall get a nice box of Faber's drawing pencils; I really need them," said Amy decidedly.

"Mother didn't say anything about our money, and she won't wish us to give up everything. Let's each buy what we want, and have a little fun; I'm sure we work hard enough to earn it," cried Jo, examining the heels of her shoes in a gentlemanly manner.

"I know I do—teaching those tiresome children nearly all day, when I'm longing to enjoy myself at home," began Meg, in the complaining tone again.

"You don't have half such a hard time as I do," said Jo. "How would you like to be shut up for hours with a nervous, fussy old lady, who keeps you trotting, is never satisfied, and worries you till you're ready to fly out the window or cry?"

"It's naughty to fret, but I do think washing dishes and keeping things tidy is the worst work in the world. It makes me cross, and my hands get so stiff, I can't practice well at all." And Beth looked at her rough hands with a sigh that any one could hear that time.

"I don't believe any of you suffer as I do," cried Amy, "for you don't have to go to school with impertinent girls, who plague you if you don't know your lessons, and laugh at your dresses, and label your father if he isn't rich, and insult you when your nose isn't nice."

"If you mean libel, I'd say so, and not talk about labels, as if Papa was a pickle bottle," advised Jo, laughing.

"I know what I mean, and you needn't be statirical about it. It's proper to use good words, and improve your vocabilary," returned Amy, with dignity.

"Don't peck at one another, children. Don't you wish we had the money Papa lost when we were little, Jo? Dear me! How happy and good we'd be, if we had no worries!" said Meg, who could remember better times.

"You said the other day you thought we were a deal happier than the King children, for they were fighting and fretting all the time, in spite of their money."

"So I did, Beth. Well, I think we are. For though we do have to work, we make fun of ourselves, and are a pretty jolly set, as Jo would say."

"Jo does use such slang words!" observed Amy, with a reproving look at the long figure stretched on the rug.

Jo immediately sat up, put her hands in her pockets, and began to whistle.

"Don't, Jo. It's so boyish!"

"That's why I do it."

"I detest rude, unladylike girls!"

  "quotes": [
    {
      "id": 0,
      "text": "\"Christmas won't be Christmas without any presents,\"",
      "beginIndex": 0,
      "endIndex": 52,
      "beginToken": 0,
      "endToken": 10,
      "beginSentence": 0,
      "endSentence": 0,
      "speaker": "Jo",
      "canonicalSpeaker": "Jo"
    },
    {
      "id": 1,
      "text": "\"It's so dreadful to be poor\"",
      "beginIndex": 84,
      "endIndex": 113,
      "beginToken": 19,
      "endToken": 27,
      "beginSentence": 1,
      "endSentence": 1,
      "speaker": "Meg",
      // W R O N G:
      "canonicalSpeaker": "Faber"
    },
    {
      "id": 2,
      "text": "\"I don't think it's fair for some girls to have plenty of pretty things, and other girls nothing at all,\"",
      "beginIndex": 157,
      "endIndex": 262,
      "beginToken": 38,
      "endToken": 63,
      "beginSentence": 2,
      "endSentence": 2,
      // W R O N G:
      "speaker": "Meg",
      // W R O N G:
      "canonicalSpeaker": "Faber"
    },
    {
      "id": 3,
      "text": "\"We've got Father and Mother, and each other,\"",
      "beginIndex": 304,
      "endIndex": 350,
      "beginToken": 73,
      "endToken": 85,
      "beginSentence": 3,
      "endSentence": 3,
      "speaker": "Beth",
      "canonicalSpeaker": "Beth"
    },
    {
      "id": 4,
      "text": "\"We haven't got Father, and shall not have him for a long time.\"",
      "beginIndex": 511,
      "endIndex": 575,
      "beginToken": 116,
      "endToken": 133,
      "beginSentence": 4,
      "endSentence": 4,
      // W R O N G:
      "speaker": "She",
      // W R O N G:
      "canonicalSpeaker": "her"
    },
    {
      "id": 5,
      "text": "\"perhaps never,\"",
      "beginIndex": 591,
      "endIndex": 607,
      "beginToken": 138,
      "endToken": 142,
      "beginSentence": 5,
      "endSentence": 5,
      "speaker": "Meg",
      // W R O N G:
      "canonicalSpeaker": "her"
    },
    {
      "id": 6,
      "text": "\"You know the reason Mother proposed not having any presents this Christmas was because it is going to be a hard winter for everyone; and she thinks we ought not to spend money for pleasure, when our men are suffering so in the army. We can't do much, but we can make our little sacrifices, and ought to do it gladly. But I am afraid I don't,\"",
      "beginIndex": 750,
      "endIndex": 1093,
      "beginToken": 174,
      "endToken": 251,
      "beginSentence": 6,
      "endSentence": 8,
      "speaker": "Meg",
      // W R O N G:
      "canonicalSpeaker": "Faber"
    },
    {
      "id": 7,
      "text": "\"But I don't think the little we should spend would do any good. We've each got a dollar, and the army wouldn't be much helped by our giving that. I agree not to expect anything from Mother or you, but I do want to buy Undine and Sintran for myself. I've wanted it so long,\"",
      "beginIndex": 1182,
      "endIndex": 1456,
      "beginToken": 270,
      "endToken": 336,
      "beginSentence": 9,
      "endSentence": 12,
      // W R O N G:
      "speaker": "Jo",
      "canonicalSpeaker": "Jo"
    },
    {
      "id": 8,
      "text": "\"I planned to spend mine in new music,\"",
      "beginIndex": 1486,
      "endIndex": 1525,
      "beginToken": 345,
      "endToken": 355,
      "beginSentence": 13,
      "endSentence": 13,
      "speaker": "Beth",
      "canonicalSpeaker": "Beth"
    },
    {
      "id": 9,
      "text": "\"I shall get a nice box of Faber's drawing pencils; I really need them,\"",
      "beginIndex": 1616,
      "endIndex": 1688,
      "beginToken": 375,
      "endToken": 393,
      "beginSentence": 14,
      "endSentence": 14,
      "speaker": "Amy",
      "canonicalSpeaker": "Amy"
    },
    {
      "id": 10,
      "text": "\"Mother didn't say anything about our money, and she won't wish us to give up everything. Let's each buy what we want, and have a little fun; I'm sure we work hard enough to earn it,\"",
      "beginIndex": 1709,
      "endIndex": 1892,
      "beginToken": 398,
      "endToken": 444,
      "beginSentence": 15,
      "endSentence": 16,
      "speaker": "Jo",
      // W R O N G:
      "canonicalSpeaker": "her"
    },
    {
      "id": 11,
      "text": "\"I know I do—teaching those tiresome children nearly all day, when I'm longing to enjoy myself at home,\"",
      "beginIndex": 1961,
      "endIndex": 2065,
      "beginToken": 459,
      "endToken": 483,
      "beginSentence": 17,
      "endSentence": 17,
      "speaker": "Meg",
      "canonicalSpeaker": "Meg"
    },
    {
      "id": 12,
      "text": "\"You don't have half such a hard time as I do,\"",
      "beginIndex": 2108,
      "endIndex": 2155,
      "beginToken": 493,
      "endToken": 507,
      "beginSentence": 18,
      "endSentence": 18,
      "speaker": "Jo",
      // W R O N G:
      "canonicalSpeaker": "her"
    },
    {
      "id": 13,
      "text": "\"How would you like to be shut up for hours with a nervous, fussy old lady, who keeps you trotting, is never satisfied, and worries you till you're ready to fly out the window or cry?\"",
      "beginIndex": 2165,
      "endIndex": 2349,
      "beginToken": 511,
      "endToken": 554,
      "beginSentence": 19,
      "endSentence": 19,
      "speaker": "Jo",
      // W R O N G:
      "canonicalSpeaker": "her"
    },
    {
      "id": 14,
      "text": "\"It's naughty to fret, but I do think washing dishes and keeping things tidy is the worst work in the world. It makes me cross, and my hands get so stiff, I can't practice well at all.\"",
      "beginIndex": 2350,
      "endIndex": 2535,
      "beginToken": 555,
      "endToken": 600,
      "beginSentence": 20,
      "endSentence": 21,
      "speaker": "Beth",
      // W R O N G:
      "canonicalSpeaker": "Faber"
    },
    {
      "id": 15,
      "text": "\"I don't believe any of you suffer as I do,\"",
      "beginIndex": 2618,
      "endIndex": 2662,
      "beginToken": 619,
      "endToken": 632,
      "beginSentence": 23,
      "endSentence": 23,
      "speaker": "Amy",
      // W R O N G:
      "canonicalSpeaker": "Faber"
    },
    {
      "id": 16,
      "text": "\"for you don't have to go to school with impertinent girls, who plague you if you don't know your lessons, and laugh at your dresses, and label your father if he isn't rich, and insult you when your nose isn't nice.\"",
      "beginIndex": 2674,
      "endIndex": 2890,
      "beginToken": 636,
      "endToken": 687,
      "beginSentence": 23,
      "endSentence": 23,
      "speaker": "Amy",
      // W R O N G:
      "canonicalSpeaker": "Faber"
    },
   {
      "id": 17,
      "text": "\"If you mean libel, I'd say so, and not talk about labels, as if Papa was a pickle bottle,\"",
      "beginIndex": 2891,
      "endIndex": 2982,
      "beginToken": 688,
      "endToken": 713,
      "beginSentence": 24,
      "endSentence": 24,
      "speaker": "Jo",
      // W R O N G:
      "canonicalSpeaker": "her"
    },
    {
      "id": 18,
      "text": "\"I know what I mean, and you needn't be statirical about it. It's proper to use good words, and improve your vocabilary,\"",
      "beginIndex": 3005,
      "endIndex": 3126,
      "beginToken": 719,
      "endToken": 748,
      "beginSentence": 25,
      "endSentence": 26,
      "speaker": "Amy",
      "canonicalSpeaker": "Amy"
    },
    {
      "id": 19,
      "text": "\"Don't peck at one another, children. Don't you wish we had the money Papa lost when we were little, Jo? Dear me How happy and good we'd be, if we had no worries\"",
      "beginIndex": 3155,
      "endIndex": 3317,
      "beginToken": 755,
      "endToken": 797,
      "beginSentence": 27,
      "endSentence": 29,
      "speaker": "Meg",
      "canonicalSpeaker": "Meg"
    },
    {
      "id": 20,
      "text": "\"You said the other day you thought we were a deal happier than the King children, for they were fighting and fretting all the time, in spite of their money.\"",
      "beginIndex": 3361,
      "endIndex": 3519,
      "beginToken": 807,
      "endToken": 841,
      "beginSentence": 30,
      "endSentence": 30,
      "speaker": "Meg",
      "canonicalSpeaker": "Meg"
    },
    {
      "id": 21,
      "text": "\"So I did, Beth. Well, I think we are. For though we do have to work, we make fun of ourselves, and are a pretty jolly set, as Jo would say.\"",
      "beginIndex": 3520,
      "endIndex": 3661,
      "beginToken": 842,
      "endToken": 882,
      "beginSentence": 31,
      "endSentence": 33,
      // W R O N G:
      "speaker": "Papa",
      "canonicalSpeaker": "Unknown"
    },
    {
      "id": 22,
      "text": "\"Jo does use such slang words\"",
      "beginIndex": 3662,
      "endIndex": 3692,
      "beginToken": 883,
      "endToken": 890,
      "beginSentence": 34,
      "endSentence": 34,
      "speaker": "Amy",
      "canonicalSpeaker": "Amy"
    },
    {
      "id": 23,
      "text": "\"Don't, Jo. It's so boyish\"",
      "beginIndex": 3845,
      "endIndex": 3872,
      "beginToken": 924,
      "endToken": 934,
      "beginSentence": 36,
      "endSentence": 37,
      // W R O N G:
      "speaker": "Jo",
      "canonicalSpeaker": "her"
    },
    {
      "id": 24,
      "text": "\"That's why I do it.\"",
      "beginIndex": 3873,
      "endIndex": 3894,
      "beginToken": 935,
      "endToken": 943,
      "beginSentence": 37,
      "endSentence": 37,
      "speaker": "Jo",
      "canonicalSpeaker": "her"
    },
    {
      "id": 25,
      "text": "\"I detest rude, unladylike girls\"",
      "beginIndex": 3895,
      "endIndex": 3928,
      "beginToken": 944,
      "endToken": 951,
      "beginSentence": 38,
      "endSentence": 38,
      // W R O N G:
      "speaker": "her",
      "canonicalSpeaker": "Unknown"
    }
  ]
}

The command to get it run:

wget -q --post-data "\"Christmas won't be Christmas without any presents,\" grumbled Jo, lying on the rug. \"It's so dreadful to be poor\" sighed Meg, looking down at her old dress. \"I don't think it's fair for some girls to have plenty of pretty things, and other girls nothing at all,\" added little Amy, with an injured sniff. \"We've got Father and Mother, and each other,\" said Beth contentedly from her corner. The four young faces on which the firelight shone brightened at the cheerful words, but darkened again as Jo said sadly, \"We haven't got Father, and shall not have him for a long time.\" She didn't say \"perhaps never,\" but each silently added it, thinking of Father far away, where the fighting was. Nobody spoke for a minute; then Meg said in an altered tone, \"You know the reason Mother proposed not having any presents this Christmas was because it is going to be a hard winter for everyone; and she thinks we ought not to spend money for pleasure, when our men are suffering so in the army. We can't do much, but we can make our little sacrifices, and ought to do it gladly. But I am afraid I don't,\" and Meg shook her head, as she thought regretfully of all the pretty things she wanted. \"But I don't think the little we should spend would do any good. We've each got a dollar, and the army wouldn't be much helped by our giving that. I agree not to expect anything from Mother or you, but I do want to buy Undine and Sintran for myself. I've wanted it so long,\" said Jo, who was a bookworm. \"I planned to spend mine in new music,\" said Beth, with a little sigh, which no one heard but the hearth brush and kettle-holder. \"I shall get a nice box of Faber's drawing pencils; I really need them,\" said Amy decidedly. \"Mother didn't say anything about our money, and she won't wish us to give up everything. Let's each buy what we want, and have a little fun; I'm sure we work hard enough to earn it,\" cried Jo, examining the heels of her shoes in a gentlemanly manner. \"I know I do—teaching those tiresome children nearly all day, when I'm longing to enjoy myself at home,\" began Meg, in the complaining tone again. \"You don't have half such a hard time as I do,\" said Jo. \"How would you like to be shut up for hours with a nervous, fussy old lady, who keeps you trotting, is never satisfied, and worries you till you're ready to fly out the window or cry?\" \"It's naughty to fret, but I do think washing dishes and keeping things tidy is the worst work in the world. It makes me cross, and my hands get so stiff, I can't practice well at all.\" And Beth looked at her rough hands with a sigh that any one could hear that time. \"I don't believe any of you suffer as I do,\" cried Amy, \"for you don't have to go to school with impertinent girls, who plague you if you don't know your lessons, and laugh at your dresses, and label your father if he isn't rich, and insult you when your nose isn't nice.\" \"If you mean libel, I'd say so, and not talk about labels, as if Papa was a pickle bottle,\" advised Jo, laughing. \"I know what I mean, and you needn't be statirical about it. It's proper to use good words, and improve your vocabilary,\" returned Amy, with dignity. \"Don't peck at one another, children. Don't you wish we had the money Papa lost when we were little, Jo? Dear me How happy and good we'd be, if we had no worries\" said Meg, who could remember better times. \"You said the other day you thought we were a deal happier than the King children, for they were fighting and fretting all the time, in spite of their money.\" \"So I did, Beth. Well, I think we are. For though we do have to work, we make fun of ourselves, and are a pretty jolly set, as Jo would say.\" \"Jo does use such slang words\" observed Amy, with a reproving look at the long figure stretched on the rug. Jo immediately sat up, put her hands in her pockets, and began to whistle. \"Don't, Jo. It's so boyish\" \"That's why I do it.\" \"I detest rude, unladylike girls\"" '0.0.0.0:9000/?properties={"annotators":"tokenize,ssplit,pos,lemma,ner,depparse,coref,quote","outputFormat":"json"}' -O -

stanfordnlp / CoreNLP

CoreNLP's quote attribution is barely usable/has very low accuracy #1090