guansss / pixi-live2d-display

A PixiJS plugin to display Live2D models of any kind.
https://guansss.github.io/pixi-live2d-display/
MIT License
870 stars 132 forks source link

About transitioning to V.0.3.0 #15

Closed yeemachine closed 3 years ago

yeemachine commented 3 years ago

Thanks again for the work on implementing the Cubism 4.0 support. Super excited to try this new version. I know it's still labeled as beta so might not be quite ready yet, but I couldn't help myself trying the update. Been able to load both 2.0 and 4.0 models so far without a problem!

What is the recommended way to update individual params like in ver 0.2.2? I've noted that the old method of saving the original update function and then calling each time on the update function no longer works on 0.3.0

  const updateFn = model.internal.motionManager.update;

  model.internal.motionManager.update = () => {
    updateFn.call(model.internal.motionManager);
    // overwrite the parameter after calling original update function
    model.internal.coreModel.setParamFloat("PARAM_ANGLE_X", mouthValue * 30);
    model.internal.coreModel.setParamFloat("PARAM_ANGLE_Y", mouthValue * 30);
    model.internal.coreModel.setParamFloat("PARAM_ANGLE_Z", mouthValue * 30);

and that internal has now been renamed internalModel?

guansss commented 3 years ago

Glad to hear that!

Yes, it's internalModel now, and the motionManager.update() now accepts arguments, so the code will be like:

const updateFn = model.internalModel.motionManager.update;

model.internalModel.motionManager.update = (...args) => {
  updateFn.apply(model.internalModel.motionManager, args);

  // ...
}

I'm still looking for a proper way to support param customizations. Will also write a guide on migrating to v0.3.0 very soon.

yeemachine commented 3 years ago

Looking forward to the guide! I'm not sure if you've tested other models yet from the sample model list, but I've run into some loading errors with them.

For example, some Cubism 4.0 versions of the models, like Shizuku or Tororo cats from the sample site give back this error.

cubismrenderer_webgl.ts:1049 Uncaught TypeError: Cannot read property 'getClippingManager' of undefined
    at ct.setupShaderProgram (cubismrenderer_webgl.ts:1049)
    at Mt.drawMesh (cubismrenderer_webgl.ts:1928)
    at Mt.doDrawModel (cubismrenderer_webgl.ts:1860)
    at Mt.drawModel (cubismrenderer.ts:32)
    at Ct.draw (Cubism4InternalModel.ts:206)
    at v._render (Live2DModel.ts:328)
    at v.e.render (Container.js:505)
    at e.render (Container.js:510)
    at r.render (Renderer.js:375)
    at t.kn.render (Application.js:97)

And if I load a model with expressions, like the Izumi model's runtime files I do get this error.

cubismexpressionmotion.ts:40 Uncaught (in promise) TypeError: Cannot read property 'length' of undefined
    at Function.create (cubismexpressionmotion.ts:40)
    at D.createExpression (Cubism4ExpressionManager.ts:34)
    at D.init (ExpressionManager.ts:45)
    at new D (Cubism4ExpressionManager.ts:18)
    at K.init (Cubism4MotionManager.ts:39)
    at new K (Cubism4MotionManager.ts:32)
    at new Ct (Cubism4InternalModel.ts:61)
    at Object.createInternalModel (factory.ts:37)
    at Live2DFactory.ts:184
    at Generator.next (<anonymous>)

That said, if it helps, both the "Haru_Greeter" model and "Hiyori" model load properly.

guansss commented 3 years ago

Whoops, I did have a plan to make a full test on various Cubism 4 models, but didn't expect problems to show up so soon... I'll look into them later.

guansss commented 3 years ago

The guide has been finished, it's rather a changelog though.

yeemachine commented 3 years ago

Thanks! Will take another pass at my project if anything needs changing since I do pass a json file modified with data uris when loading a user uploaded live2D model. By the way, I never did understand how to pass in new parameters to update with this new method?

You said something like this? how is the args structured?

const updateFn = model.internalModel.motionManager.update;

model.internalModel.motionManager.update = (...args) => {
  updateFn.apply(model.internalModel.motionManager, args);

  // ...
}

How exactly, let's say for example, could I manually update an eye param updated? Like this?

//old way
model.internal.coreModel.setParamFloat("PARAM_EYE_L_OPEN", .5);
//new way?
model.internalModel.motionManager.update = () => {
  updateFn.apply(model.internalModel.motionManager, {
 "PARAM_EYE_L_OPEN": .5
  });
}
guansss commented 3 years ago

The way to modify params didn't change. The arguments of motionManager.update() are not noteworthy in this case, they are just transfered to the original function as is.

const updateFn = model.internalModel.motionManager.update;

model.internalModel.motionManager.update = (...args) => {
  updateFn.apply(model.internalModel.motionManager, args);

  model.internalModel.coreModel.setParamFloat("PARAM_ANGLE_X", mouthValue * 30);
  model.internalModel.coreModel.setParamFloat("PARAM_ANGLE_Y", mouthValue * 30);
  model.internalModel.coreModel.setParamFloat("PARAM_ANGLE_Z", mouthValue * 30);
}

Loading models from data URLs will possibly be a built-in feature at some point.

yeemachine commented 3 years ago

Looks like it's different functions and parameter strings for Cubism 4.0 vs 2.0

// 4.0
model.internalModel.coreModel.setParameterValueById("ParamAngleX", 30); 
//2.0
model.internalModel.coreModel.setParamFloat("PARAM_ANGLE_X", 30); 

I've gotten most of the manual param manipulation linked up now!

The one thing that isn't quite working is the ability to overwrite the blinking function in Cubism 4.0 I passed a blank function for Cubism 2.0 model eyeBlink and it allows me to overrride the default blinking function, but that doesn't seem to behave the same for the 4.0 models.

 model.internalModel.eyeBlink.update = () => {}

 const updateFn = model.internalModel.motionManager.update; 

 model.internalModel.motionManager.update = (...args) => {

      updateFn.apply(model.internalModel.motionManager, args);

        if(typeof model.internalModel.coreModel.setParameterValueById === "function"){
          //cubism 4.0
          model.internalModel.coreModel.setParameterValueById("ParamAngleX", 20); 
          model.internalModel.coreModel.setParameterValueById("ParamEyeLOpen", .4);
          model.internalModel.coreModel.setParameterValueById("ParamEyeROpen", .4);
      }else if (typeof model.internalModel.coreModel.setParamFloat === "function"){
          //cubism 2.0
          model.internalModel.coreModel.setParamFloat("PARAM_ANGLE_X", 20); 
          model.internalModel.coreModel.setParamFloat("PARAM_EYE_L_OPEN",.4);
          model.internalModel.coreModel.setParamFloat("PARAM_EYE_R_OPEN", .4);
      }
}
guansss commented 3 years ago

The blinking function on Cubism 4 is eyeBlink.updateParameters(), you can override that one.

Additionally, you can actually return true in motionManager.update() to prevent the eyeBlink from being updated:

model.internalModel.motionManager.update = (...args) => {
  updateFn.apply(model.internalModel.motionManager, args);

  // do stuff...

  // this prevents blinking
  return true;
}
guansss commented 3 years ago

0.3.0-beta.1 has been released. You can now modify the params by listening to one of these update events: beforeMotionUpdate, afterMotionUpdate, beforeModelUpdate.

Here is the equivalent of previous workaround:

model.internalModel.on('afterMotionUpdate', function() {
  // `this` references to `model.internalModel`
  this.coreModel.setParamFloat('PARAM_MOUTH_OPEN_Y', mouthValue);
})

Also, eyeBlink is now an optional property, you can fully disable it by setting model.internalModel.eyeBlink = undefined.

yeemachine commented 3 years ago

Many thanks for the release. All the demo models I've found online (and even some datamined ones) seem to load perfectly, and the updated method of setting parameters manually is much cleaner!

yeemachine commented 3 years ago

This is unrelated, but are there any ways of contacting your about other questions, like Live2D ones, that might not suit a Github issues thread? A twitter or something?

guansss commented 3 years ago

Sure, a good place to have conversations would be the discussions, which is opened for this purpose exactly.