Cannot extract more than 2 hand landmark sets

[cm-yamamoto-hiroki]

Hello, how can I detect the landmark sets for more than 2 hands using Multi Hand Tracking? I could receive only 2 landmark sets.

• What I did
  • Installed MediaPipe
  • Changed settings. multi_hand_detection_cpu.pbtxt 208: max_vec_size: 2 -> max_vec_size: 8 multi_hand_tracking_desktop_live.pbtxt 22: min_size: 2 -> min_size: 8
  • Input a video file in which 4 hands are recorded.
• Result
  • Good:
  • 4 rects covering hands are shown, so hand detection is correct.
  • Bad:
  • The landmark sets for only 2 hands are shown.
  • The hands are not sticked to specific hands in frames. ex) hand1, hand2 -> hand1, hand3 -> ...
• Environment Windows 10, WSL Ubuntu 18.04
• Supplement I traced the process in MultiHandLandmarkSubgraph and HandLandmarkSubgraph.
  • BeginLoopCalculator received a packet that included the 4 rects of the hand detection result.
  • ImagePropertiesCalculator::Process(=ImageCropping) was called 4 times.
  • ImageTransformationCalculator::Process was called 4 times.
  • TfLiteConverterCalculator::Process was called 4 times.
  • TfLiteInferenceCalculator::Process was called only 2 times.
  • EndLoopCalc had received only 2 packets before it received a BATCH_END packet.

[cm-yamamoto-hiroki]

The result was like this image. You can see the landmarks on the second and third hands (not on the first and 4th ones).

[cm-yamamoto-hiroki]

I've resolved this issue. I changed the InputStreamHandler of TfLiteInferenceCalculator from FixedSizeInputStreamHandler to DefaultInputStreamHandler.

In mediapipe\mediapipe\calculators\tflite\tflite_inference_calculator.cc 318:

  // Assign this calculator's default InputStreamHandler.
-  cc->SetInputStreamHandler("FixedSizeInputStreamHandler");
+  cc->SetInputStreamHandler("DefaultInputStreamHandler");

[eknight7]

Marking resolved, thanks for the update!