(JoinMessage) removeDuplicates doesn't ignore mode=ignore messages

[kharybdys]

Often we use removeDuplicates to basically do a DISTINCT in Navascript. This however doesn't work if you've used a mode="ignore" message inside the array elements to be looped when with a different structure as JoinMessage uses BaseMessageImpl's isEqual and that one considers the entire submessage structure irrespective of mode.

A workaround exists of making sure the (sub)message structure is independent of the data resulting in isEqual saying "yes" again.

[kharybdys]

Example script to replicate

<?xml version="1.0" encoding="UTF-8"?>
<navascript xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="http://www.navajo.nl/schemas/navascript.xsd" debug="response">

    <!-- Run this versus KNVB DB -->
    <message name="Array" type="array">
        <message name="Array" type="array_element">
            <property name="Language" type="string" value="en"/>
            <property name="Addition" type="string" value="Marte"/>
        </message>
    </message>

    <map.sqlquery datasource="'sportlinkkernel'">
        <sqlquery.query xml:space="preserve">
            SELECT ct.language
            ,      ct.codeid
            FROM   codetable ct
            WHERE  ct.typeofcode = 'GAME_TYPE'
            AND    ct.codeid IN ( 'VE', 'ZA' )
        </sqlquery.query>

        <message name="GameTypesTmp">
            <sqlquery.resultSet>
                <property name="GameType">
                    <expression value="$columnValue( 'codeid' )"/>
                </property>
                <param name="Language">
                    <expression value="$columnValue( 'language' )"/>
                </param>
                <param name="Addition">
                    <expression value="null"/>
                </param>

                <map.joinmessage>
                    <joinmessage.join message1="'/Array'" type="'outer'" ignoreSource="false"/>
                    <!-- Compare output with and without mode="ignore" -->
                    <message name="FindLanguage" mode="ignore">
                        <!-- Block that causes the wrong behaviour later on with removeDuplicates -->
                        <map ref="resultMessage" filter="[/@Language] == $property( 'Language' )">
                            <param name="Addition">
                                <expression value="$property( 'Addition' )"/>
                            </param>
                        </map>
                        <!-- Work-around -->
<!--                         <map ref="resultMessage"> -->
<!--                             <param name="Addition" condition="[/@Language] == $property( 'Language' )"> -->
<!--                                 <expression value="$property( 'Addition' )"/> -->
<!--                             </param> -->
<!--                         </map> -->
                    </message>
                </map.joinmessage>
            </sqlquery.resultSet>
        </message>
    </map.sqlquery>

    <map.joinmessage>
        <joinmessage.join message1="'/GameTypesTmp'" type="'outer'" ignoreSource="false" removeDuplicates="true"/>
        <message name="GameTypes">
            <map ref="resultMessage"/>
        </message>
    </map.joinmessage>

</navascript>

[kharybdys]

@aschoneveld I think the more basic discussion is about what mode="ignore" should mean. Is it correct that isEqual looks at it, but should it possibly be ignored only when trying to removeDuplicates? Or should it always be ignored for the isEqual check.

In my current understanding, mode="ignore" just means that said message and substructure is not serialized. But in practical terms there are a couple of moments where the "invisible" message bites us, so maybe it should be reimplemented to never become part of the navajo structure of in- or outdoc?

See also #490 and #499

[aschoneveld]

mode=ignore is deprecated by using #613