search

aorogat / Maestro

0 stars 1 forks source link

System Parameters #1

Open Reham-Osama opened 3 months ago

Reham-Osama commented 3 months ago

Hello, I have a couple of questions related to the parameters used to run the system

  1. How to specify the number of questions to generate in the benchmark?
  2. I managed to run the system with DBpedia, But I got 10 output files, is this the final output or do you have another step to merge them in one file or choose from the generated questions?

Thanks

aorogat commented 3 months ago

Hi Reham, I wanted to inform you that I made some adjustments to the system regarding the generation of multiple benchmarks, but I forgot to revert it back to its original state. Rectifying this is quite simple on your end. You can accomplish it by uncommenting lines 72 and 73 in the code found here: link to the relevant GitHub file.

As indicated in this line of code, the variable Settings.benchmarkSizeBeforePrune determines the number of questions generated before the pruning process, where we select the properties of the questions for the final benchmark.

Upon reviewing the code, I noticed that this particular process hasn't been updated. I'll be adding the necessary updates shortly by uploading the latest version of the code.

Reham-Osama commented 3 months ago

Hi Abdelgany,

I uncommented the two lines that you mentioned, but still I got numbers of questions more than what is specified in Settings.benchmarkSizeBeforePrune. (I got 1272 questions while I asked for 100) I think it is because generatedQuestions is reinitialized at every loop so in the condition Line 72, the condition is never satisfied so no break happens. I reached this conclusion because while the system is running, I keep getting this message Generated Questions tell now: 0

aorogat commented 3 months ago

You're correct, Reham. There are some additional changes because I configured it to generate multiple benchmarks, and I need to reset the code. I was attempting to update it and refresh the GitHub repository, but encountered a conflict, so I need to review it carefully. I'll try to address it in the near future. Please note that the component package requires code refactoring. I had begun refactoring the entire project but didn't finish it before starting on another project. I can send you my local version by email until I update the GitHub repository.