eMoflon / benchmarx

Infrastructure for implementing benchmarx: benchmarks for bidirectional transformation (bx) tools. Also contains a collection of example benchmarx and test runners for various and diverse bx tools.
GNU General Public License v3.0
9 stars 12 forks source link

False test failure for testIncrementalInsertsDynamicConfig #56

Closed georghinkel closed 7 years ago

georghinkel commented 7 years ago

The following two family registers are asserted as not equal according to the test framework (I have slightly improved the formatting):

Expected

FamilyRegister{
    families=[
        Family{
            familyName="Simpson",
            father=Just(FamilyMember{firstName="Homer"}),
            mother=Just(FamilyMember{firstName="Maggie"}),
            sons=[],
            daughters=[]
        },
        Family{
            familyName="Skinner",
            father=Just(FamilyMember{firstName="Seymour"}),
            mother=Nothing,
            sons=[],
            daughters=[]
        },
        Family{
            familyName="Skinner",
            father=Just(FamilyMember{firstName="Seymour"}),
            mother=Nothing,
            sons=[
                FamilyMember{firstName="Seymour"}
            ],
            daughters=[]
        },
        Family{
            familyName="Skinner",
            father=Just(FamilyMember{firstName="Seymour"}),
            mother=Nothing,
            sons=[
                FamilyMember{firstName="Seymour"}
            ],
            daughters=[]
        }
    ]
}

Actual

FamilyRegister{
    families=[
        Family{
            familyName="Simpson",
            father=Just(FamilyMember{firstName="Homer"}),
            mother=Just(FamilyMember{firstName="Maggie"}),
            sons=[],
            daughters=[]
        },
        Family{
            familyName="Skinner",
            father=Nothing,
            mother=Nothing,
            sons=[
                FamilyMember{firstName="Seymour"}
            ],
            daughters=[]
        },
        Family{
            familyName="Skinner",
            father=Just(FamilyMember{firstName="Seymour"}),
            mother=Nothing,
            sons=[],
            daughters=[]
        },
        Family{
            familyName="Skinner",
            father=Just(FamilyMember{firstName="Seymour"}),
            mother=Nothing,
            sons=[
                FamilyMember{firstName="Seymour"},
                                FamilyMember{firstName="Seymour"}
            ],
            daughters=[]
        }
    ]
}

The only difference between them is the order of the family skinner that appears multiple times. I thought that the order of the families in the family register would not be of importance. Is that correct? If so, then perhaps generating these string descriptions has some bug.

georghinkel commented 7 years ago

@anthonyanjorin @tbuchmann

tbuchmann commented 7 years ago

Can you plesae tell us, which postcondition assertion fails? As far as I can see from your code pasted above, the last family in your actual FamilyRegister has a father named Seymour and two sons with the same name. I just checked the expected models ("FamilyAfterBwdInsertion1-6.xmi") and none of them contains a Family Skinner with two sons. Maybe this is the reason why the test case fails?

anthonyanjorin commented 7 years ago

Another "diff": Your actual model has a family with no father, while all families in the expected model have fathers.

georghinkel commented 7 years ago

The postcondition that fails is line 91 util.assertPostcondition("FamilyAfterBwdInsertion5", "PersonAfterBwdInsertion5");. But yes, indeed, there is a structural difference beyond the order.

However, given that the result is correct after the postcondition in line 84, it seems my solution just picked a different family "Skinner". But ah, now I see that the one that is picked now does have a father but there is a different family that has no father currently. That explains the difference.