Constructor Arity Mismatch Preventing Case Execution

First of all, thank you for your effort for the challenge-tool, its design is very clean and makes it very convenient to run a benchmark.

I did however encounter a bug. In the executor.py class, when running a case with the run function there is an assumption that a class that corresponds to a "resource" specified in a metadata.json file has a constructor that accepts 5 arguments (not counting the self argument).

https://github.com/kg-construct/challenge-tool/blob/94fa696a24a8a937e2e9ed587f50ae9d164f81bb/bench_executor/executor.py#L389-L396

For some provided classes this is the case, see for example mysql.py and postgresql.py. This is not the case for the query.py and virtuoso.py classes:

https://github.com/kg-construct/challenge-tool/blob/94fa696a24a8a937e2e9ed587f50ae9d164f81bb/bench_executor/query.py#L18-L21

https://github.com/kg-construct/challenge-tool/blob/94fa696a24a8a937e2e9ed587f50ae9d164f81bb/bench_executor/virtuoso.py#L42-L46

This might also not be the case for the user defined class used to run their tool to partecipate in the challenge. I think that if this a limitation it should be highlighted and presented to the user in the README and the provided classes should be corrected. I noticed this problem with the virtuoso.py and query.py class, i do not know if others are affected.

How to reproduce: run any case that makes use of Virtuoso ./exectool run --runs=1 --root=downloads/eswc-kgc-challenge-2024/gtfs-madrid-bench/scale_1/ results in: Case failure due to exception: Virtuoso.__init__() takes 5 positional arguments but 6 were given

kg-construct / challenge-tool

Constructor Arity Mismatch Preventing Case Execution #2