You need to have a valid 'references' and 'system' folder under project_directory/project. Please check your rouge properties file.

[thomasmccabe]

We are experiencing this error even though:

• the rouge.properties file is properly configured

• our file names comply with the naming convention rules

• our project directory has the correct reference and system folder setup. Is there a remedy to this issue or has anyone else experienced this? Thanks!

[kavgan]

Could you please paste your directory structure as well as the command you are using to run the program. Also, the exact stacktrace would be useful.

[thomasmccabe]

Sorry, I should have included that information initially. Also - it may be worth noting that ROUGE was initially working when comparing four reference summaries to a single system summary. But once we modified our algorithm to adjust a few different parameters at a time to output multiple system summaries to evaluate against the reference summaries to optimize the algorithm we ran into this issue.

Directory Structure: Directory in rouge.properties

==Project Directory==

#root directory for system and reference summaries
project.dir=rouge_testing_input_output/project

Command Used: java -jar rouge2-1.2.jar Stacktrace 2019-04-15 14:41:49 INFO SettingsUtil:33 - Using rouge.properties='/home/tom/Senior_Design/MNRVA/rouge.properties' 2019-04-15 14:41:49 INFO ROUGECalculator:201 - Working on d30002 ngram=L 2019-04-15 14:41:49 ERROR ROUGECalculator:180 - 0 2019-04-15 14:41:49 ERROR ROUGECalculator:181 - You need to have a valid 'references' and 'system' folder under rouge_testing_input_output/project. Please check your rouge.properties file

Thank you for the help!

[thomasmccabe]

I figure out the issue - some of the system summaries contained a "_" (underscore) character that was keeping ROUGE from running and outputting a score for all the system files.

[kavgan]

😆 I was just about to respond. I'm glad you figured it out