search

eMoflon / benchmarx

Infrastructure for implementing benchmarx: benchmarks for bidirectional transformation (bx) tools. Also contains a collection of example benchmarx and test runners for various and diverse bx tools.
GNU General Public License v3.0
9 stars 12 forks source link

Please merge the FunnyQT solution #46

Closed tsdh closed 7 years ago

tsdh commented 7 years ago

Here's my FunnyQT version. I can do all the batch cases but fails for almost all the incremental ones. The incremental scenario is now what I designed FunnyQT's BX transformation API for, so that's fine. :-)

anthonyanjorin commented 7 years ago

Great! One of our goals was exactly that one can filter for only the tests that make sense for a specific tool, so nice to see that this works.

Please consider adding and documenting the results for FunnyQT here: https://github.com/eMoflon/benchmarx/blob/master/examples/familiestopersons/BenchmarxFamiliesToPersons/results/TestResults.xlsx This way others can understand why certain tests are failing (expected limitation vs. error).

tsdh commented 7 years ago

Ok, will do.

tsdh commented 7 years ago

@anthonyanjorin In the Excel sheet, what does "runtime" mean in the tool column headers?

anthonyanjorin commented 7 years ago

the values refer to the bx tool features from our paper: http://ceur-ws.org/Vol-1827/paper6.pdf (the feature model was also in the case description).

runtime as opposed to fixed refers to the type of update policy (configuration) the tool supports. runtime means you can make decisions and state preferences during the synchronisation and not only at design time (I believe your solution is runtime).

tsdh commented 7 years ago

I don't quite understand. Right now, the preferences (parents over children, existing families over new) are given as arguments to the transformation and the relations are coded in a way that acknowledges these preferences. So this has been done by me at design time.

Instead of providing just booleans, you could also have a higher-order transformation (not in the sense of higher-order transformation but in the sense of higher-order functions) which receives other relations and uses them in its own relations. But still, the relations you'd pass to the transformation have been written at design time.

Well, of course these relations passed in could query the user, perform a google search, or let a neural network decide...

So I'd say that there's possibility for "runtime" but right now it is just as encoded at design time.

anthonyanjorin commented 7 years ago

A tool that supports runtime configuration obviously supports design time configuration as well (just as you explained). For our tests to run deterministically the "runtime" configuration is actually hard coded in the tests, but if this can be supplied as an extra argument (without the tool knowing what is going to be required for a certain test) then the tool must support runtime configuration in general.

Some solutions or tools, however, might require forward and backward propagation to be deterministic functions, meaning that all required decisions must be encoded at design time and that there is no means to influence decision making at runtime. This might be necessary to ensure well-behavedness (whatever that means for the specific tool), or perform certain analyses or optimisations. Such tools will simply ignore the extra argument (just like how state-based tools ignore the provided deltas).

Concrete examples: applications conditions in TGG rules to ensure that no two rules are applicable at the same time, a BiGUL implementation without preferences as an extra input argument, ...

tsdh commented 7 years ago

Ok, understood. Thank you!