Infrastructure for implementing benchmarx: benchmarks for bidirectional transformation (bx) tools. Also contains a collection of example benchmarx and test runners for various and diverse bx tools.
We have a plug and play (via remote desktop) Share virtual machine available from: http://is.ieis.tue.nl/staff/pvgorp/share/?page=ConfigureNewSession&vdi=Ubuntu12LTS_BenchmarX.vdi
git clone https://github.com/eMoflon/benchmarx.git benchmarx
Help/Eclipse market place
). Restart and import the PSF file (Import/Team/Team Project Set
) in the benchmark example folder you're interested in. As an example, for BenchmarxFamiliesToPersons this would be (benchmarx/examples/familiestopersons/projectSet.psf
). If your workspace does not compile then please create an issue for us.
For all examples, make sure you switch to UTF 8
as encoding for your Eclipse workspace (Window/Preferences/General/Workspace/Text file encoding
)./BenchmarxFamiliesToPersons/src/org/benchmarx/examples/familiestopersons/testsuite/FamiliesToPersonsTestCase.java/tools()
.FamiliesToPersonsTestCase.java/tools()
for the necessary steps. As of 10.05.2023, FunnyQT no longer works, probably due to compatibility reasons with the current Java version./BenchmarxFamiliesToPersons/results/TestResults.xlsx
./BenchmarxFamiliesToPersons/src/org/benchmarx/examples/familiestopersons/testsuite/scalability/ScalabilityMeasurements.java
as required. The referenced tools should of course be setup and working as expected in Step 4/5/6. To add an implementation of an existing benchmarx example, e.g., FamiliesToPersons
with your favourite bx tool, do the following (taking FamiliesToPersons
as an example, substitute as appropriate for another example):
/examples/familiestopersons/BenchmarxFamiliesToPersons/src/org/benchmarx/examples/familiestopersons/implementations/<your_bx_tool>
.emoflon/EMoflonFamiliesToPersons.java
and medini/MediniQVTFamiliesToPersons.java
. If your bx tool does not use EMF, then take a look at bigul/BiGULFamiliesToPersons.java
./examples/familiestopersons/implementationsArtefacts
./examples/familiestopersons/BenchmarxFamiliesToPersons/src/org/benchmarx/examples/familiestopersons/testsuite/FamiliesToPersonsTestCase.java
to include your new test runner.bigul/README-SETUP
). /examples/familiestopersons
. This consists of a folder metamodels
for the source and target metamodels in an EMF representation, a folder BenchmarxFamiliesToPersons
for the actual testsuite, and a folder implementationArtefacts
for any further implementation artefacts.BenchmarxFamiliesToPersons
is an Eclipse project that can be run as a JUnit test suite and contains all test cases as JUnit tests, all resources (input and expected models), any implementations of the benchmarx example, and auxiliary classes for both metamodels used to simplify the comparison of output and expected models (or model representations).