Allow to connect Key Points

[samirbr]

Is your feature request related to a problem? Please describe. There is a case where the Key Points might be related (i.e. same person Nose, Hand). It would be easier to relate different Key Points to a same "entity" (i.e. same person) if they are connected.

Describe the solution you'd like Add a tag to enable Key Point connection (i.e.KP Node to KP Ear) and Key Point connection schema.

Possible implementation?

<View>
  <KeyPointLabels name="tag" toName="img" strokewidth="5" connected="true">
    <Label value="Nose" background="green"></Label>
    <Label value="Ear" background="blue">
      <Joint connectTo="Nose" />
    </Label>
    <Label value="Lip" background="red">
       <Joint connectTo="Nose" />
    </Label>
  </KeyPointLabels>
  <Image name="img" value="$image" zoom="true"></Image>
</View>

[deppp]

Hey @samirbr,

Thanks for bringing that up! I think it can also be relevant for other labeling types, and the question we need to answer is how to put in a group of labels (possibly connected somehow) into the object (image).

<KyPointLabels name="kp">
  <Label value="Nose" />
  <Label value="Ear" />
  <Label value="Lip" />
  <LabelGroup value="Nose and Ear" connected="True" values="Nose,Ear" states="[[1,1], [2,2]" />
  <LabelGroup value="Nose and Ear and Lip" connected="True" values="Nose,Ear,Lip" states="[[1,1], [2,2], [3,3]]" connections="[[0,1],[0,2]]" />
<KeyPointLabels>

That would provide labels that can create 2 & 3 keypoints with the initial state being the location of the keypoints on the image. And for three keypoint label nodes are going to be connected using the line. I think it's a general approach that would enable you to create multiple elements at once, and also extendable further if we introduce an ability to ask for the initial state from the user/model.

Overall it's in the pipeline, but need to think more about implementation, ideas are more than welcome.

[iitdpooja]

Hey,

I need to annotate video using keypoints. I am also looking for solution like this. Can somebody please explain to me about this. I want to make a skeleton structure using 100 keypoints.

[makseq]

Hi, LS doesn't support skeleton structure with keypoints yet, but you can try keypoints grouping using the region panel. https://www.youtube.com/watch?v=V3ieJGky3yc

[iitdpooja]

Hey,

Does label-studio support "keypoints grouping using the region panel" using Videos??? I have lots of videos and I have to start with 20fps.

Thanks and Regards, Pooja Kumari

On Mon, May 9, 2022 at 11:50 PM Max Tkachenko @.***> wrote:

Hi, LS doesn't support skeleton structure with keypoints yet, but you can try keypoints grouping using the region panel. https://www.youtube.com/watch?v=V3ieJGky3yc

— Reply to this email directly, view it on GitHub https://github.com/heartexlabs/label-studio/issues/141#issuecomment-1121617702, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQB6DY52Z4KSLC4EIMGYQLLVJGCA3ANCNFSM4JVNTDMQ . You are receiving this because you commented.Message ID: @.***>

[makseq]

In video labeling you can use only bounding boxes for now. Yes, you will be able group bboxes using the region panel.