google-ai-edge / mediapipe

Cross-platform, customizable ML solutions for live and streaming media.
https://ai.google.dev/edge/mediapipe
Apache License 2.0
27.6k stars 5.16k forks source link

pass multi varibale from java to graph(.binarypb) #1714

Closed milad-4274 closed 3 years ago

milad-4274 commented 3 years ago

Hi, I have five button in android and I want to pass boolean value to graph as input stream, my approach for adding one button to MainActivity.java is inspired from faceeffect MainActivity.java. I'e defined an input stream name variable: private static final String IS_LIP_SELECTED_INPUT_STREAM_NAME ="is_lip_selected"; I've defined an object and a boolean variable and a view: private final Object isLipSelectedLock = new Object(); private boolean isLipSelected; private View createEffectLipView; I'e wrote a function for adding view and assign it to view variable: createEffectLipView = createEffectLipViewFunc();

private View createEffectLipViewFunc() {
      Button createEffectLipView = new Button(getApplicationContext());
      LayoutParams layoutParams=new LayoutParams(300, 150);
      createEffectLipView.setLayoutParams(layoutParams);

        createEffectLipView.setText("Eye line");
        createEffectLipView.setBackgroundColor(Color.parseColor("#7b71d9"));
        createEffectLipView.setGravity(Gravity.CENTER );
        createEffectLipView.setPadding(450, 0, 50, 50);
        createEffectLipView.setTextColor(Color.parseColor("#ffffff"));
        createEffectLipView.setTextSize((float) 10);

    return createEffectLipView;
 }

and added my created view to viewgroup: viewGroup.addView(createEffectLipView); fuction to change boolean value by clicking on button:

  createEffectLipView.setOnClickListener(new View.OnClickListener() {
    @Override
    public void onClick(View v) {
        Log.i("lip selected","jyvfui");
        switchLipEffect();
    }
});

and set the view visible: createEffectLipView.setVisibility(View.VISIBLE); and finally add the boolean value to graph as input_stream:

    processor.setOnWillAddFrameListener(
      (timestamp) -> {
        Packet isLipSelectedPacket = null;
        try {
          synchronized (isLipSelectedLock) {
            isLipSelectedPacket =
                processor.getPacketCreator().createBool(isLipSelected);
          }

          processor
              .getGraph()
              .addPacketToInputStream(
                IS_LIP_SELECTED_INPUT_STREAM_NAME,
                isLipSelectedPacket,
                  timestamp);
        } catch (RuntimeException e) {
          Log.e(
              TAG,
              "milad Exception while adding packet to input stream while switching effects: " + e);
        } catch (Exception e){
          Log.e(
            TAG,
            "milad exception " + e
          );
        } finally {
          if (isLipSelectedPacket != null) {
            isLipSelectedPacket.release();
          }
        }
      });

and there are 4 other buttons exactly similar to this button. but only one input passed to graph. and other seems empty. is there any limitation for input_stream in method setOnWillAddFrameListener or other used methods? is there better way to handle multi input_streams and pass it to graph? thank you.

milad-4274 commented 3 years ago

My problem solved. I used multi processor.setOnWillAddFrameListener and solved by use one setOnWillAddFrameListener and add 5 packets into it.