align modules for purposes of sign comparison

[kvesik]

In order for signs to be compared in an intuitive/effective manner (see issue #330 ), the modules in those signs need to be aligned such that "similar" modules are compared to each other wherever possible. This issue deals specifically with the alignment task.

Kathleen says: I also discovered that Yurika and I had laid out a really clever way of actually trying to align elements in signs (i.e. putting elements next to each other in the expansion tree), to handle all of the issues with respect to timing and x-slots, and trying to be maximally useful phonologically. This is instead of just matching elements by their labels. I need to think through still how to make this work for relation and non-manual modules, but here’s the basic idea:

The basic principle of alignment is to maximize similarity between signs. Note that I am proposing here that we NEVER align modules by virtue of their timing -- we align them based on other characteristics, and then check the matching of their timing.

1. Align the “sign type” modules of each sign and compare them. There can only be one sign type per sign, so this is straightforward.

2. Movement modules: i. First 'align by hand.' That is, try to align hand1 modules from sign1 to hand1 modules from sign2. If sign1 and sign2 each have only hand1 modules, or each have only hand2 modules, then proceed to the next step of alignment. If sign1 has both hand1 and hand2 modules, while sign2 has only hand1 modules (or vice versa), then align the hand1 modules only, and leave all hand2 modules unmatched. If sign1 has only hand1 modules, and sign2 has only hand2 modules, then this is the only time that non-matching hand modules can be aligned. ii. After aligning by hand as described above, try to align by movement type (perceptual shape, joint specific, or handshape change) -- e.g., if sign 1 has both perceptual shape movement and joint-specific movement, while sign 2 has only joint-specific movement, align the two joint-specific movements, and then say that sign 1 has an extra perceptual shape movement that doesn’t have a match. If both signs have two of the same type of movements (e.g. two perceptual shapes), move down to the top-most characteristic (e.g., what the perceptual shape or the joint-specific movement is, like 'straight' or 'close/open'), and align ones that match at that level. If things can't be aligned based on any of the above, align by coding order (e.g. align sign 1’s H1.Mov3 with sign 2’s H1.Mov3, regardless of content).

3. Location modules: i. First ‘align by hand’ as for the movement module. ii. After aligning by hand, try to align by general location type (body-anchored or signing space) -- e.g., if sign1 has both body-anchored and signing space locations, and sign2 has only a body-anchored location, then align the body-anchored modules and leave the signing-space module unmatched. If there are multiple locations of the same type (e.g., multiple body-anchored locations), use the uppermost (in the tree) location specifications to align (e.g., align two head-locations rather than a head location with a torso location, if possible). If modules still can't be aligned, align by coding order.

4. Absolute orientation modules: i. First ‘align by hand’ as for the movement module. ii. After aligning by hand, align by palm orientation if possible (e.g. align two palm-up modules). If not possible, align by finger root direction. If still not possible, align by coding order.

5. Hand configuration modules: i. First ‘align by hand’ as for the movement module. ii. After aligning by hand, align by handshape name if possible (e.g. align two '5' handshapes). If not possible, align by coding order. [NB: this one might eventually need to get refined.]

6. Relation modules: I need to figure these out!

7. Non-manual modules: I need to figure these out!

Originally posted by @kchall in https://github.com/PhonologicalCorpusTools/SLPAA/issues/330#issuecomment-2316661748

[kvesik]

@kchall when doing 'align by hand', what would you like to do about non-hand articulators? Do you ever want to try and align (eg) a movement module articulated by H1 with a movement module articulated by A1?

[kchall]

@kvesik Ooh, great question; this was definitely designed before we even thought of the other articulator options. Prioritizing "Align by hand" in the algorithm above does have the effect that even similar articulations on different hands are not aligned, so I think we should carry that through. But, I think it's okay if could end up aligning across articulators at the 'end' of the algorithm. That is, we basically do:

1. Align by hand
2. Align by arm
3. Align by leg
4. Align by number

...so in the same way that Hand 1 modules can get aligned with Hand 2 modules in the rare event that one sign has only Hand 1 and the other has only Hand 2, then it could be that Hand 1 modules in sign 1 are aligned with Arm 1 modules in sign 2 if these are the only modules of the type in each sign.

That is, I'd rather try to compare across articulators than leave things completely unaligned, but it should be the last-case scenario.

I guess we have to decide about a priority within hypothetical cross-articulator comparisons; I think I'd do hand-arm first, then hand-leg, and then arm-leg.

Does this make sense?

[kvesik]

Notes from 20241122 meeting:

Re aligning by hand: If any modules are marked as "both hands", then match those up where possible. But if sign1 has a "both hands" module and sign2 has a H1 module, for example, then it's reasonable to align those since "both" includes H1.