Slevinski / apitxt

The ApiTxt Project is a custom build environment for the development of SignPuddle 3 and other RESTful APIs.
https://SignPuddle.com
2 stars 2 forks source link

[Question] Separating symbols for the two hands #2

Closed andry-tino closed 5 years ago

andry-tino commented 5 years ago

I am conducting a research project about the possibility to reconstruct the full sign flow from the single knowledge of the sign flow coming from one hand. The v0.1 release has the research overview paper you can read to get more info 😄

Question

What I need to do is basically building a dictionary og signs where I can extract the dominant hand flow. I had a look at the JSON being returned by the REST API and there seems not to be such information captured there: only the full flow can be found, which means I will manually need to add these entries.

  1. Can you confirm this?
  2. Any pointer to ease the process?
    • Fx: could you suggest me the 200 most frequent signs I can start from?

Thanks

andry-tino commented 5 years ago

Also... it might be good for you to set up GitHub bug templates so you can make an immediate dinstinction between bugs and questions 😄

Slevinski commented 5 years ago

Interesting project. One thought. The dominant hand is mostly signer and context specific. In general, a right-handed signer will use their right hand as the dominant hand for signing, but not always. Similarly, a left-handed signer will use their left hand as the dominant hand for signing, but not always. So the term "dominant hand" is not synonymous for right hand and the same hand is not always dominant.

With SignWriting, a sign is written in two-parts. The temporal prefix is a sequential list of symbols. The spatial signbox is a two-dimensional arrangement of symbols. The spatial signbox is used to display a sign for reading. The temporal prefix is used for sorting and analysis. Screen Shot 2019-06-18 at 9 51 34 AM

I have a presentation which covers this topic called "A sign by any other name". https://www.slideshare.net/StephenSlevinski/a-sign-by-any-other-name

For SignWriting handshape symbols, it is not possible to tell which is a right-hand and which is a left-hand. Some symbols can be either depending on context. For movement symbols, it is possible to tell the difference between right-hand and left-hand depending on the arrow-head fill pattern.

To extract the dominant hand information from the spatial signbox is difficult if not impossible. However, it is feasible to extract the dominant hand information from a properly constructed temporal prefix. Consider this sign... Screen Shot 2019-06-18 at 9 52 26 AM

The SignSpelling Sequence is Valerie's creation of how a temporal prefix should be constructed. It is primarily used for sorting dictionaries. It was tested in small groups with good results. The SignSpelling Sequence starts with the dominant hand, followed by the non-dominant hand. Then the movement of the dominant hand, followed by the non-dominant hand. This pattern continues until the handshapes and movements are completed. The end of the SignSpelling Sequence is used for locations and headshapes.

Alternatively, there is a project call SignTyp, which is a linguistic coding system. It uses SignWriting, a notation system, as input and analyzes the temporal prefix to default an analysis of the sign. This analysis is verified and corrected by a trained linguist. Screen Shot 2019-06-18 at 10 11 40 AM

This information is captured in a comma delimited list. It contains the symbol key using Formal SignWriting in ASCII (FSW), followed by the symbol type initial (A for action, H for handshape), followed by articulator for handedness (L for left, R for right, B for both, and N for neutral, then contains numbers for sequence and subsequence within the sign. Screen Shot 2019-06-18 at 10 12 02 AM

You can find more information about the SignTyp project by Rachel Channon online. https://signtyp.uconn.edu/

andry-tino commented 5 years ago

As always, thanks for the very well written clarification and the very nice answer. Yes I understand your point about the dominant hand and the dominant-hand is a concept I have not associated to a specific body side in my research effort 😄

About extracting the right hand info, I think I will need to somewhat manually extract such flow. I am in a rush because an hackathon starts tomorrow and I need to produce results by Friday. So for now I will do this manually with a limited set of signs just in order to be able to verify some mathematical equations I have devised to prove that ASL can be reconstructed from the dominant flow. It is going to be an approximation, but good material to build up a research effort on a longer term.

I might come up with questions but one thing is for sure, you are going to be mentioned a lot in my paper :)

andry-tino commented 5 years ago

Closing as this pretty much answers the question