vitruv-tools / Vitruv

View-based Development and Model Consistency Framework
http://vitruv.tools
Eclipse Public License 1.0
14 stars 19 forks source link

Print Models with Newlines #349

Closed jGleitz closed 3 years ago

jGleitz commented 3 years ago

This PR changes DefaultModelPrinter such that it prints models in multiple lines, e.g. like this:

        Resource(
                uri=file:/home/josh/.eclipse/workspaces/junit-workspace/Vitruv/ComponentEquivalenceTest/creating%20a%20component/PCM%20→%20%7BJava%7D/src/testRepository/testComponent/TestComponentImpl.java,
                content=[CompilationUnit(
                        name="testRepository.testComponent.TestComponentImpl.java",
                        namespaces=["testRepository", "testComponent"],
                        imports=[],
                        classifiers=[Class(
                                name="TestComponentImpl",
                                typeParameters=[],
                                members=[],
                                defaultMembers=[],
                                annotationsAndModifiers=[public],
                                implements=[],
                                extends=∅,
                                defaultExtends=java.lang.Object
                        )]
                )]
        )

Since this PR makes the model printing facility even more complicated than it was before, it might be worthwhile to reflect on whether it is reasonable to spend so much code & energy on a test helper. I think it is.

In our model-based test, we have a strong trade-off between creating tests that are easy to use and tests that are easy to maintain:

With our current tools, we either do coarse matching (which gives unhelpful error messages) or write very specific assertions (which gives long, hard-to-maintain-code). From my point of view, this dilemma can be improved upon with well-formatted model prints. My idea is to only specify the expected model in tests (without writing a separate assertion for every relevant feature). This is yields easy-to-understand and easy-to-maintain code. However, it means that developers will be confronted with whole model trees when a test fails. This quickly becomes very hard to understand. That is where model printing comes in: my wager is that, with easy-to-read prints of models, the error messages become helpful enough such that the tests are reasonably easy to use.

Having that in mind, the model printing should also be adaptable to the specifics of domains. Writing a customized printer for a domain, on the other hand, should also be reasonably simple. One should not be confronted with implementation details. That is why I chose a relatively involved approach for model printing: By separating different aspects into separate types, it should be simple to make customizations without breaking the output.

I hope that makes sense!

HeikoKlare commented 3 years ago

This trade-off discussion and the conclusions that you make sound reasonable to me. One thing that came into my mind to possibly improve failure cause detection could be the use of EMFCompare. It is the purpose of that tool to find model differences, such that it could be used for both the validation of test results as well as for the detection of the cause for a failure. I.e., when EMFCompare considers an expected and an actual model as equal, tests would succeed and in other cases it could provide the comparison result. For cases in which that comparison is not useful, the model printing capabilities could still be useful.

jGleitz commented 3 years ago

True, it looks like ModelTreeEqualityMatcher duplicates what is already implemented with EMF Compare. I’ll have a look at whether we can use EMF Compare instead.