guansss / pixi-live2d-display

A PixiJS plugin to display Live2D models of any kind.
https://guansss.github.io/pixi-live2d-display/
MIT License
870 stars 132 forks source link

Memory leak when setting too many parameters too quickly? #9

Closed yeemachine closed 3 years ago

yeemachine commented 3 years ago

Hi again. I've been having some performance issues in the Live2D face tracked web app I'm currently developing. I am currently updating the Live2D parameters as shown below how you've demoed in a pen in the past, but it seems that after a couple of minutes on my iphone/mac, the CPU will start spiking in temp, throttle the frame rate for a couple seconds, and then go back to normal, before eventually forcing a page refresh.

The page still runs fine with both the Live2D model and Tensorflow face tracking running concurrently but stutters when the values are being updated in the update functions.

Any clue on if this is a Live2D inherent issue? or if there's a better way to set these params? Or if inherent animations within Live2D models will cause conflicts with setParamFloat? The output from face capture is fairly noisy, and does update every animationFrame.

    //overwrite blink function
    model.internal.eyeBlink.update = () => {}

    //saves original update function
    const updateFn = model.internal.motionManager.update; 

    model.internal.motionManager.update = () => {
      // overwrite the parameter after calling original update function

      updateFn.call(model.internal.motionManager);

      model.internal.coreModel.setParamFloat("PARAM_ANGLE_X", $trackedStats.head.x); 
      model.internal.coreModel.setParamFloat("PARAM_ANGLE_Y", $trackedStats.head.y); 
      model.internal.coreModel.setParamFloat("PARAM_ANGLE_Z", $trackedStats.head.z);

      model.internal.coreModel.setParamFloat("PARAM_BODY_ANGLE_Z", $trackedStats.head.z*.3);
      model.internal.coreModel.setParamFloat("PARAM_BODY_ANGLE_X", $trackedStats.head.x*.3); 

      model.internal.coreModel.setParamFloat("PARAM_MOUTH_OPEN_Y", $trackedStats.mouth.y); 
      model.internal.coreModel.setParamFloat('PARAM_MOUTH_FORM', .2 + .8 * $trackedStats.mouth.x);

      model.internal.coreModel.setParamFloat("PARAM_EYE_L_OPEN", $trackedStats.eye.l);
      model.internal.coreModel.setParamFloat("PARAM_EYE_R_OPEN", $trackedStats.eye.r);
      model.internal.coreModel.setParamFloat("PARAM_EYE_BALL_X", $trackedStats.pupil.x); 
      model.internal.coreModel.setParamFloat("PARAM_EYE_BALL_Y", $trackedStats.pupil.y);
      // model.internal.coreModel.setParamFloat("PARAM_EYE_BALL_FORM", 0); //gangimari

      // model.internal.coreModel.setParamFloat("PARAM_BROW_L_Y", 0);
      // model.internal.coreModel.setParamFloat("PARAM_BROW_R_Y", 0);

      // model.internal.coreModel.setParamFloat("PARAM_CHEEK", 0); //blush
    };
guansss commented 3 years ago

No idea, I can't seem to reproduce this problem, could you provide a minimal demo for reproduction?

yeemachine commented 3 years ago

I couldn't reproduce it either with a minimal demo, using Math.random on the param values. Must be the amount of memory the facial tracking takes. I'll close this for now.