search

kavgan / ROUGE-2.0

ROUGE automatic summarization evaluation toolkit. Support for ROUGE-[N, L, S, SU], stemming and stopwords in different languages, unicode text evaluation, CSV output.
https://kavgan.github.io/ROUGE-2.0
Apache License 2.0
210 stars 37 forks source link

trouble with specifying ngram= L, SU4 #8

Closed justine747 closed 6 years ago

justine747 commented 6 years ago

rouge 2.0 calculator works fine when screenshots.docx n-gram size assigned is an integer value (1,2,3) but throws exception in java main class when ngram is assigned a string value , eg. ngram=L,SU4 (in the rouge.properties folder). Any clue how to go about this error?

RxNLP commented 6 years ago

I am going to take a look at this today.

On Tue, Apr 3, 2018 at 2:33 AM, justine747 notifications@github.com wrote:

rouge 2.0 calculator works fine when screenshots.docx https://github.com/RxNLP/ROUGE-2.0/files/1870740/screenshots.docx n-gram size assigned is an integer value (1,2,3) but throws exception in java main class when ngram is assigned a string value , eg. ngram=L,SU4 (in the rouge.properties folder). Any clue how to go about this error?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/RxNLP/ROUGE-2.0/issues/8, or mute the thread https://github.com/notifications/unsubscribe-auth/AKUNIsqjqExfOY9B7z-GmsSVYkUeFKT2ks5tkzPhgaJpZM4TEqS7 .

RxNLP commented 6 years ago

So, you will need to use ROUGE2 version 1.2 and above for this to work. https://github.com/RxNLP/ROUGE-2.0/tree/master/versions

The older versions of ROUGE2 only supports ROUGE-N and not L and SU4.

Let me know if you have trouble with the latest version.

On Tue, Apr 3, 2018 at 2:33 AM, justine747 notifications@github.com wrote:

rouge 2.0 calculator works fine when screenshots.docx https://github.com/RxNLP/ROUGE-2.0/files/1870740/screenshots.docx n-gram size assigned is an integer value (1,2,3) but throws exception in java main class when ngram is assigned a string value , eg. ngram=L,SU4 (in the rouge.properties folder). Any clue how to go about this error?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/RxNLP/ROUGE-2.0/issues/8, or mute the thread https://github.com/notifications/unsubscribe-auth/AKUNIsqjqExfOY9B7z-GmsSVYkUeFKT2ks5tkzPhgaJpZM4TEqS7 .

justine747 commented 6 years ago

Thanks for replying and providing your insight. However i would like to point that i am having difficulty running v1.2 as it always gives path error. It somehow fails to recognise that valid reference and system folder has been given under projects/test-summarization. screenshots_2nd.docx

RxNLP commented 6 years ago

Can you try editing the rouge.properties file and set the absolute path to your summarization project under project.dir. Let me know if this works. If not, can you send me your test files and I will try to reproduce the problem?

#==Project Directory==
#root directory for system and reference summaries
project.dir=projects/test-summarization

On Thu, Apr 5, 2018 at 12:34 PM, justine747 notifications@github.com wrote:

Thanks for replying and providing your insight. However i would like to point that i am having difficulty running v1.2 as it always gives path error. It somehow fails to recognise that valid reference and system folder has been given under projects/test-summarization. screenshots_2nd.docx https://github.com/RxNLP/ROUGE-2.0/files/1881235/screenshots_2nd.docx

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/RxNLP/ROUGE-2.0/issues/8#issuecomment-379034868, or mute the thread https://github.com/notifications/unsubscribe-auth/AKUNIvLmyEkJ2bEQbvBcSTRzSfbBW34Mks5tlmOxgaJpZM4TEqS7 .

justine747 commented 6 years ago

​ ROUGE-2.0-master.zip https://drive.google.com/file/d/1PtSq7JFA-F2Y02F3J-3tTwSrwRIN2FAy/view?usp=drive_web ​I have tried specifying the absolute path still shows the same error. I have attached screenshots. Tell me if am specifying the wrong way? Isn't it unix style format? Nevertheless I have attached the test files also. My references and summary is in the ROUGE-2.0-master\versions\v1.2\rouge2-1.2-runnable\projects\test-summarization directory. Plz check whether it works on your system.

On Fri, Apr 6, 2018 at 2:45 AM, rxnlp notifications@github.com wrote:

Can you try editing the rouge.properties file and set the absolute path to your summarization project under project.dir. Let me know if this works. If not, can you send me your test files and I will try to reproduce the problem?

#==Project Directory==
#root directory for system and reference summaries
project.dir=projects/test-summarization

On Thu, Apr 5, 2018 at 12:34 PM, justine747 notifications@github.com wrote:

Thanks for replying and providing your insight. However i would like to point that i am having difficulty running v1.2 as it always gives path error. It somehow fails to recognise that valid reference and system folder has been given under projects/test-summarization. screenshots_2nd.docx https://github.com/RxNLP/ROUGE-2.0/files/1881235/screenshots_2nd.docx

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/RxNLP/ROUGE-2.0/issues/8#issuecomment-379034868, or mute the thread https://github.com/notifications/unsubscribe-auth/ AKUNIvLmyEkJ2bEQbvBcSTRzSfbBW34Mks5tlmOxgaJpZM4TEqS7

.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/RxNLP/ROUGE-2.0/issues/8#issuecomment-379079299, or mute the thread https://github.com/notifications/unsubscribe-auth/AkRV7LLg8IcQR_eqq5A3WwBlzNFMMpxnks5tlomBgaJpZM4TEqS7 .

RxNLP commented 6 years ago

Couple of things:

  1. there was a small bug causing the error for your files, this version should have resolved it: https://github.com/RxNLP/ROUGE-2.0/tree/master/versions/v1.2.1
  2. Your formatting of input and output is incorrect for this package. Please follow the formatting guidelines here: https://github.com/RxNLP/ROUGE-2.0/blob/master/docs/usage-documentation.md . The results would be unreliable if the formatting is in html.

On Thu, Apr 5, 2018 at 10:25 PM, justine747 notifications@github.com wrote:

​ ROUGE-2.0-master.zip https://drive.google.com/file/d/1PtSq7JFA-F2Y02F3J- 3tTwSrwRIN2FAy/view?usp=drive_web ​I have tried specifying the absolute path still shows the same error. I have attached screenshots. Tell me if am specifying the wrong way? Isn't it unix style format? Nevertheless I have attached the test files also. My references and summary is in the ROUGE-2.0-master\versions\v1.2\rouge2-1.2-runnable\ projects\test-summarization directory. Plz check whether it works on your system.

On Fri, Apr 6, 2018 at 2:45 AM, rxnlp notifications@github.com wrote:

Can you try editing the rouge.properties file and set the absolute path to your summarization project under project.dir. Let me know if this works. If not, can you send me your test files and I will try to reproduce the problem?

#==Project Directory==
#root directory for system and reference summaries
project.dir=projects/test-summarization

On Thu, Apr 5, 2018 at 12:34 PM, justine747 notifications@github.com wrote:

Thanks for replying and providing your insight. However i would like to point that i am having difficulty running v1.2 as it always gives path error. It somehow fails to recognise that valid reference and system folder has been given under projects/test-summarization. screenshots_2nd.docx <https://github.com/RxNLP/ROUGE-2.0/files/1881235/screenshots_2nd.docx

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/RxNLP/ROUGE-2.0/issues/8#issuecomment-379034868, or mute the thread https://github.com/notifications/unsubscribe-auth/ AKUNIvLmyEkJ2bEQbvBcSTRzSfbBW34Mks5tlmOxgaJpZM4TEqS7

.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/RxNLP/ROUGE-2.0/issues/8#issuecomment-379079299, or mute the thread https://github.com/notifications/unsubscribe-auth/AkRV7LLg8IcQR_ eqq5A3WwBlzNFMMpxnks5tlomBgaJpZM4TEqS7 .

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/RxNLP/ROUGE-2.0/issues/8#issuecomment-379144505, or mute the thread https://github.com/notifications/unsubscribe-auth/AKUNIq-OPoN36I01oTGb7pR2X6IHrksYks5tlu4egaJpZM4TEqS7 .

justine747 commented 6 years ago

This version did resolve my problem. Also I have incorporated changes in the formatting of input and output.

Thanks a lot for your time and assistance all along. Truly appreciate your help :)

On Fri, Apr 6, 2018 at 10:50 PM, rxnlp notifications@github.com wrote:

Couple of things:

  1. there was a small bug causing the error for your files, this version should have resolved it: https://github.com/RxNLP/ROUGE-2.0/tree/master/versions/v1.2.1
  2. Your formatting of input and output is incorrect for this package. Please follow the formatting guidelines here: https://github.com/RxNLP/ROUGE-2.0/blob/master/docs/usage-documentation.md . The results would be unreliable if the formatting is in html.

On Thu, Apr 5, 2018 at 10:25 PM, justine747 notifications@github.com wrote:

​ ROUGE-2.0-master.zip https://drive.google.com/file/d/1PtSq7JFA-F2Y02F3J- 3tTwSrwRIN2FAy/view?usp=drive_web ​I have tried specifying the absolute path still shows the same error. I have attached screenshots. Tell me if am specifying the wrong way? Isn't it unix style format? Nevertheless I have attached the test files also. My references and summary is in the ROUGE-2.0-master\versions\v1.2\rouge2-1.2-runnable\ projects\test-summarization directory. Plz check whether it works on your system.

On Fri, Apr 6, 2018 at 2:45 AM, rxnlp notifications@github.com wrote:

Can you try editing the rouge.properties file and set the absolute path to your summarization project under project.dir. Let me know if this works. If not, can you send me your test files and I will try to reproduce the problem?

#==Project Directory==
#root directory for system and reference summaries
project.dir=projects/test-summarization

On Thu, Apr 5, 2018 at 12:34 PM, justine747 notifications@github.com wrote:

Thanks for replying and providing your insight. However i would like to point that i am having difficulty running v1.2 as it always gives path error. It somehow fails to recognise that valid reference and system folder has been given under projects/test-summarization. screenshots_2nd.docx <https://github.com/RxNLP/ROUGE-2.0/files/1881235/ screenshots_2nd.docx

— You are receiving this because you commented. Reply to this email directly, view it on GitHub <https://github.com/RxNLP/ROUGE-2.0/issues/8#issuecomment-379034868 , or mute the thread https://github.com/notifications/unsubscribe-auth/ AKUNIvLmyEkJ2bEQbvBcSTRzSfbBW34Mks5tlmOxgaJpZM4TEqS7

.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/RxNLP/ROUGE-2.0/issues/8#issuecomment-379079299, or mute the thread https://github.com/notifications/unsubscribe-auth/AkRV7LLg8IcQR_ eqq5A3WwBlzNFMMpxnks5tlomBgaJpZM4TEqS7 .

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/RxNLP/ROUGE-2.0/issues/8#issuecomment-379144505, or mute the thread https://github.com/notifications/unsubscribe-auth/AKUNIq- OPoN36I01oTGb7pR2X6IHrksYks5tlu4egaJpZM4TEqS7

.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/RxNLP/ROUGE-2.0/issues/8#issuecomment-379319616, or mute the thread https://github.com/notifications/unsubscribe-auth/AkRV7DjAYNumnQHrNQ79lY-NQ6vZ0N0Lks5tl6PYgaJpZM4TEqS7 .

RxNLP commented 6 years ago

That's good to know! Happy evaluating :-)

On Sat, Apr 7, 2018 at 4:47 AM, justine747 notifications@github.com wrote:

This version did resolve my problem. Also I have incorporated changes in the formatting of input and output.

Thanks a lot for your time and assistance all along. Truly appreciate your help :)

On Fri, Apr 6, 2018 at 10:50 PM, rxnlp notifications@github.com wrote:

Couple of things:

  1. there was a small bug causing the error for your files, this version should have resolved it: https://github.com/RxNLP/ROUGE-2.0/tree/master/versions/v1.2.1
  2. Your formatting of input and output is incorrect for this package. Please follow the formatting guidelines here: https://github.com/RxNLP/ROUGE-2.0/blob/master/docs/ usage-documentation.md . The results would be unreliable if the formatting is in html.

On Thu, Apr 5, 2018 at 10:25 PM, justine747 notifications@github.com wrote:

​ ROUGE-2.0-master.zip https://drive.google.com/file/d/1PtSq7JFA-F2Y02F3J- 3tTwSrwRIN2FAy/view?usp=drive_web ​I have tried specifying the absolute path still shows the same error. I have attached screenshots. Tell me if am specifying the wrong way? Isn't it unix style format? Nevertheless I have attached the test files also. My references and summary is in the ROUGE-2.0-master\versions\v1.2\rouge2-1.2-runnable\ projects\test-summarization directory. Plz check whether it works on your system.

On Fri, Apr 6, 2018 at 2:45 AM, rxnlp notifications@github.com wrote:

Can you try editing the rouge.properties file and set the absolute path to your summarization project under project.dir. Let me know if this works. If not, can you send me your test files and I will try to reproduce the problem?

#==Project Directory==
#root directory for system and reference summaries
project.dir=projects/test-summarization

On Thu, Apr 5, 2018 at 12:34 PM, justine747 < notifications@github.com> wrote:

Thanks for replying and providing your insight. However i would like to point that i am having difficulty running v1.2 as it always gives path error. It somehow fails to recognise that valid reference and system folder has been given under projects/test-summarization. screenshots_2nd.docx <https://github.com/RxNLP/ROUGE-2.0/files/1881235/ screenshots_2nd.docx

— You are receiving this because you commented. Reply to this email directly, view it on GitHub <https://github.com/RxNLP/ROUGE-2.0/issues/8# issuecomment-379034868 , or mute the thread https://github.com/notifications/unsubscribe-auth/ AKUNIvLmyEkJ2bEQbvBcSTRzSfbBW34Mks5tlmOxgaJpZM4TEqS7

.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <https://github.com/RxNLP/ROUGE-2.0/issues/8#issuecomment-379079299 , or mute the thread https://github.com/notifications/unsubscribe-auth/AkRV7LLg8IcQR_ eqq5A3WwBlzNFMMpxnks5tlomBgaJpZM4TEqS7 .

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/RxNLP/ROUGE-2.0/issues/8#issuecomment-379144505, or mute the thread https://github.com/notifications/unsubscribe-auth/AKUNIq- OPoN36I01oTGb7pR2X6IHrksYks5tlu4egaJpZM4TEqS7

.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/RxNLP/ROUGE-2.0/issues/8#issuecomment-379319616, or mute the thread https://github.com/notifications/unsubscribe- auth/AkRV7DjAYNumnQHrNQ79lY-NQ6vZ0N0Lks5tl6PYgaJpZM4TEqS7 .

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/RxNLP/ROUGE-2.0/issues/8#issuecomment-379460588, or mute the thread https://github.com/notifications/unsubscribe-auth/AKUNIljRtMyKJv711NZ0B0lp8AUc2z9gks5tmJlBgaJpZM4TEqS7 .