Art: Main character / Drop Hunter ('Scilly')

Vianvolaeus commented 3 years ago

This will serve as the design issue for the main character, user of the 'megasword'.

Will update as appropriate.

The idea of using the ears from #1294, but with the existing character design, was explored briefly.

Vianvolaeus commented 3 years ago

Iris sizing was the same. If we want to go smaller, that's possible. Same for rotation.

avaer commented 3 years ago

Note that the changes were done from overlaying the 3d shot that was edited rather than the 2d concept, since that has an expression it's not usable to overlay and fit the shapes.

Then we will need to do the proportions again to better match, because we're not that close yet.

avaer commented 3 years ago

Here's my attempt at what I think would be the neutral pose:

avaer commented 3 years ago

https://user-images.githubusercontent.com/6926057/128095757-de1fd96f-d028-4fe5-9cc8-0f8f9cfe9470.mp4

@Vianvolaeus working on an inverse emote hack that would allow us to have a new default, plus also keep the old morphs by simultaneously inverting the default.

Vianvolaeus commented 3 years ago

Sock refit

New hair, with rig

avaer commented 3 years ago

I think we're close to done on the design, the above looks great.

I've been posing test scenes to get strong emotes out and it's working well, though there are some places where the clothes clip a bit, like the underarm of the jacket.

avaer commented 3 years ago

This is nearly done, I think with @plankatron's animations it is already largely unblocked.

If it's time consuming, rather than finishing the morph set, we could start on the other characters.

Todo

[x] ~~Bake Face.M_F00_000_00_Fcl_MTH_Down = 0.1~~ It shouldn't be default, it is too easy to add after. After a lot of testing, it doesn't look bad at the default.
[ ] Small clothes clipping for relatively normal bone folds

[ ] Morph degbugging (time consuming):
- [ ] Bite mark doesn't generally track
- [ ] "fung"/"fang" resets the eyes
- [ ] most expressions are blowing geometry limits (I think we should be able to fix a lot of this by moving vertices around)

avaer commented 3 years ago

Checking VRChat:

[x] Let's keep both eyes open and only apply the closed eye morph in the scenes we need it in, in Blender. Otherwise due to mirrored eyes in VR, it's possible to experience the Abraham Lincoln effect when the direction changes. This is, I take it, an expensive morph remapping procedure.
[x] Boots should close off at the top or else you can look in to see floating feet.
[x] Hair should be rigged
[ ] When sitting, the shirt and skirt clip with the legs. It's not easy but we should weight paint so typical sit positions work
[x] The legs height doesn't seem to be aligned for VR. It seems the eyes are too high, so the character is always on toes in a normal standing VR position
[x] We should get the sword on an emote -- hopefully keeping the animation
[ ] If/when https://github.com/webaverse/app/issues/1407 is modeled, we can add that in as well

All of these things gradually add value to the model.

avaer commented 3 years ago

VRChat test album taken by Jin: https://imgur.com/a/1HiZDoV

avaer commented 3 years ago

The photo shoot also makes the case for a hue rotation pass to see what other colors would work better. The yellow/blue is kind of biting and muddied in warm tones like this.

Vianvolaeus commented 3 years ago

Keep eyes open (...) w/ closed eye morph seperate

We're setup with FaceIt rigging on this character now - shapekeys, standard rig and control rig for animation. This step includes a shapekey for the single closed left eye, and has the characters' face with wide open eyes by default.

@plankatron, this is currently a separate armature object using a Child Of constraint to make it move with the head. I'll let you decide how you'd like to set it up yourself for animation - other than that, it's been left as is.

As this is done, I've added in the bitemark again too, which is dependent on shapekeys being final.

Boots should close off at the top

Done! This also includes a pass to reduce body geo and close off other clothing areas as necessary.

Hair should be rigged

Hair itself is rigged, but will need config in the VRM stage to add collisions and physics bone components.

Sitting causes clipping

This is largely improved - this one's a particular headache due to the rotation required - it is offset from the leg bone, so we cannot parent bones to it effectively to rotate that area of the jacket. That said - it's largely improved, and also changed the skirt weighting to provide a bit more modesty when seated.

VR alignment

This is down to VRchat's look position in their avatar descriptor - it was likely offset from the correct eye position. Also relevant is the user's own world scaling (real world height setting). This can throw off the scaling and positioning.
Note that they're recently upgraded to a new version of unity, we should expect bugs with their avatar behaviour in near future - still a decent testing ground nonetheless.

Sword on emote

This is possible, but last in prio

Morpher

Dependant on #1407. Should be easy enough to add on completion.

Hue rotation pass

Color random pass here:

Vianvolaeus commented 3 years ago

Blend uploaded for the rigged and corrected version. I highly advise a review of this before we do the VRM process. There are some instructions included for controlling each armature individually. The face control rig is pretty intuitive, but LMK if there are any issues with it. Direct shapekey control can still be used as before, too.

avaer commented 3 years ago

Thanks for the update, awesome work @Vianvolaeus.

Hair itself is rigged, but will need config in the VRM stage to add collisions and physics bone components.

It comes down to bone names I believe; it might just work if we try it.

Also thanks for the color splash, I think there are some interesting ideas even if we don't do them yet. I think keeping the jacket muted + bright highlights is in general the way to go. I checkmarked some colors that I think are leaning in the right direction but I don't think we need any changes yet.

I highly advise a review of this before we do the VRM process.

Yup I'll do that.

madjin commented 3 years ago

Scilly pureref: https://cdn.discordapp.com/attachments/641036402829819914/875776072170504212/scillia.pur

I understand things may change, good base to start from though. I want to add in a section for WIP later.

avaer commented 3 years ago

We got the face of this animated with Faceit + ARKit.

It's quite good, though some morphs collide with each other on this first pass.

I wasn't able to get head tracking out of it, but the iOS apps can do it relatively well. One that can do it is https://www.facemotion3d.info/, but it performs quite badly.

Also, the face capture morphs make the eyes really stand out as quite large/surprised/froglike. This suits the design and it does not look bad, but we might be able to get more emotion out of a more standard eye baseline. That needs more experimentation.

https://cdn.discordapp.com/attachments/831974724510810172/875898334668259448/Blender__C__Users_avaer_Downloads_Blender_Addon_and_Sample_Head_20210802_FM_Blender_sampleHead_T1.bl.mp4

I also managed to get the face animation exported into blende, where it acts as a regular track in case we need it. That is in the GDrive.

avaer commented 3 years ago

From Faceit/ARKit testing:

[x] Experiment with getting head/neck rotations working
[ ] Experiment with tighter eyes for expressivity of emotions like forrowed brows

avaer commented 3 years ago

I got the head working by enabling it the ARKit settings, and then regenerating the routes. The neck might need IK.

avaer commented 3 years ago

I got the neck working by adding a softer rotation constraint (0.2), similar to how it's done in the engine. It's good enough that the brain buys it at least.

avaer commented 3 years ago

Trying the latest model, it does not export top GLB correctly. It could be a problem with Blender but we need to handle it.

It is not working in Blender with an export -> import either.

Blender 2.92

Vianvolaeus commented 3 years ago

Looks like this is probably a matter of correct export settings. by default GLB exports seem to export and blend the vertex colors as part of the base color (really not helpful since they're used for an ID map for baking...unusual default settings, I can't think of another format that does this!)

Looks like it's exporting all objects in the scene potentially as well. I'll take a look and find what's problematic, and best export settings for what we need here, so we can save an operator preset in Blender.

avaer commented 3 years ago

Thanks, though I don’t know 100% if we can change the settings or set an operator. This is happening via plugin too, so the plugin would need to have this setting set, and that setting would need to work for all models.

Vianvolaeus commented 3 years ago

Firstly, under Include dropdown - easiest way to do this IMO is to Limit to: Selected Objects. Simply exports what you have selected in the viewport before moving to export. (This would remove the Cube object)

Under Geometry dropdown: Vertex Colors should be disabled on export unless specifically needed or intentionally set. (This would stop the blue color being added)

(tl;dr on their usage in gltf spec:)

Transparent materials seem to work best set to Alpha Clip in Blender (still need to troubleshoot / wait on GLB exporter fix for proper transparency, I believe)

Ambient Occlusion may forcibly multiply in some viewers (babylon.js does by default, but can be toggled. Webaverse seems to directly read the ORM and it's forced on, even if not attached directly as an AO texture, but as roughness/metal).

I've updated the AO in the ORM Map as well to fix the mesh matching for the bake ('2d' brows were projecting onto the face, creating incorrect occlusion)

(This would fix the multiplied shadows in the model eg: face).

Note: this has been substantially more of a pain to redo than it should be as a result of the back and forth on facial geo, effectively having to reconstruct some geo to match the original painting mesh, since things were changed halfway through, furthering my opinion that orthos for proportioning confirmation before modelling is a good idea.

We should probably disable Shape Key Normals on export!

from an artistic standpoint - these file up file size and can make facial morphs look bad. In this case, since it's got a lot of blendshapes, it adds something ridiculous like 50MB on to the file size.

The technicals are a little over my head, but from my understanding this might even be necessary if we're doing facecap: https://github.com/KhronosGroup/glTF-Blender-IO/issues/959#issuecomment-594278416 (:

Preview in weba & babylon:

Vianvolaeus commented 3 years ago

settings would need to work for all models.

Selected Models can be circumvented by just having your export scene only have precisely what you want to export in it

Vertex Colors can be manually cleared, but they're used for baking an ID map at present. It's a holdover that isn't a problem in any other file format AFAIK (and was kept since we've been going back and forth on materials a bunch - didn't make sense to clear them)

madjin commented 3 years ago

The VRChat version of Scilly has dynamic bones + working visemes, there's a version with parent constraints + megasword too.

https://user-images.githubusercontent.com/32600939/130494832-d1ed202e-1107-4147-a5ea-10779d461676.mp4

Via will be doing VRM export to test in the Webaverse app.

Vianvolaeus commented 3 years ago

Matched up the two versions, so there's now a version with the correct visemes / dyn bones, gestures (face gestures, not visemes) as well as a parent constraints for the sword (back / hand toggle). Couldn't get parent constraints working in the original one for whatever reason (literally identical to the setup Jin has). Unity is an absolute pain for things like this that are 'simple'.

There's also a VRM that's got full ARkit shapes setup properly, as well as hair bones / collisions. Looks fine visually pre-mint, but unable to fully test in Webaverse. Still functioning fine in other VRM apps.

Added seperate folders for final VRMs and VRchat packages to the drive:

GaladWarder commented 3 years ago

@Vianvolaeus would it be possible to either make a version that responds to lighting, or even better, show me how to tweak the nodes to make it non-emissive? Looking to use these VRM's for the illustrations, but need the lighting info.

If you just gave me a quick overview or a screenshot of the way the nodes need to be changed, I can figure it out from there

Vianvolaeus commented 3 years ago

The .blend master files are set up in this way, so you should be able to use those. The VRMs have a toon shader on that'll be emissive.

Drop Hunter

https://drive.google.com/file/d/1SyWKUm-pUWAMwUkcbswGXFaXZpytTz1Q/view?usp=sharing

Jun

https://drive.google.com/file/d/1yx6eiLEEbaUhpujIMHhORxhnbLAE59l8/view?usp=sharing

GaladWarder commented 3 years ago

Ah, awesome. I must have just grabbed the wrong file; I went for the VRM. Thank you!

avaer commented 3 years ago

I think we need to fix the collar a bit:

avaer commented 3 years ago

Note: This model does not have complete mouth speech morphs yet.

The ones that work are:

Face.M_F00_000_00_Fcl_MTH_A
Face.M_F00_000_00_Fcl_MTH_I

I believe the other shape keys just need to be implemented.

The eye geometry also has wrinkles in the angry eyes:

Vianvolaeus commented 3 years ago

We have range of speech morphs here:

If these need renaming to the above naming scheme for compatibility, that's fine, but ultimately i'd suggest support for these since it's the output from FACEIT.

The (---)EYE.Angry is incorrect though, has been fixed.

Collar was been brought down at the front to be in line with the jacket.

avaer commented 3 years ago

Yup I realized that we were just going to go with the faceit morphs, but for compatibility at least copy to the old name is probably good. File size bloat isn't a problem, we will need to seriously optimize these for engine anyway.

Vianvolaeus commented 3 years ago

Sure thing.

During VRM export, we can remove anything that's not referenced in a VRMBlendShapeClip automatically - only the referenced morphs will be retained if we choose that option.

Vianvolaeus commented 3 years ago

Suggestions were made about simplifying and redoing jacket/sock tex. What I've done to keep it interesting in 3d is add minor detail patterns to different areas that aren't too distracting from the main shape / design that was shown.

Colors are now slightly gradient - they're a bit more interesting than just flat color, but from memory they should ready as a single color if you were to draw it from memory. They're also more saturated, as was suggested in #1483.

Couple of variations of both:

Comparison to older version

Vianvolaeus commented 3 years ago

Cloth details were added via a multires sculpt and baked in as well, much like #1386's outfit was. Worth noting if we ever want to touch up existing or purchased assets - we can add cloth detail in non-destructively to a base mesh (to a degree)

avaer commented 3 years ago

The simplified "hoodie" looks good, but can we move the pockets down? It feels like actual pockets would be lower.

For the leggings: can we do something simpler? Simple horizontal striped socks, slightly bunched at the bottom? There is a lot of complexity going on in the lower half, which can be simplified. The shoes I think would probably look fine

For the collar: although it looks good, a couple of artists have expressed that it is hard to sketch such a complicated hexagonal pattern. 2D compatibility could be greatly improved with a simpler mat.

avaer commented 3 years ago

Taking a step back, WDYT about just in general adjusting to the 2D art? It was originally pulled from the same face (gear), and I think it's the strongest concept so far, at least in 2D, which is where the project requirements bottom out (it should be easily drawable). So the challenge would be to tweak it more faithfully back into 3D.

I think the delta is in the eyes, ensuring the hair doesn't clip with the collar (possibly thinning out the sides?), and the legs.

avaer commented 3 years ago

I could not find this model in GDrive: https://github.com/webaverse/app/issues/1382#issuecomment-921062256

avaer commented 3 years ago

Still has rigging morph problems around the waist when sitting:

Vianvolaeus commented 3 years ago

Changes were made to address the above. Material work has been done, remade the socks etc. I've edited the jacket to actually include the scarf - seems like the best solution to the problem and conforms to the proposed simplification, solves the hair clipping issues etc.

Think I accidentally updated to the wrong file yesterday, so I've added in a new one following the proposed naming scheme with the other files.

There's a shapekey on the jacket to accentuate the the lift it should have when sitting at an extreme angle, so it can easily be corrected for stills. It's as close as it can be without having extreme deformation / folding in on itself. (AFAIK we can't use volume preservation / double quaternion skinning in app, so I've foregone using the preserve volume on the Armature modifier)

avaer commented 3 years ago

Can we highlight the pocket square like in the concept?

For the legs I was thinking even simpler, it doesn't need to have a logo, or any features. What I had in mind was essentially one of two styles. Both styles are primarily driven by ease to replicate:

Short and striped

This can be cut a bit above the ankle.

Long and solid

I think what we have is really close to this.

Vianvolaeus commented 3 years ago

Can we highlight the pocket square like in the concept?

Think I preemptively screenshotted - it should be there in the blend on the drive 👍

I can cut back the details on the textures no problem, sounds good.

avaer commented 3 years ago

Thanks, I'm also curious how a stripe and making the socks shorter would work.

This turned out to be surprisingly important in cohesiveness of sitting shots.

Vianvolaeus commented 3 years ago

~~New sock variations. Updating the files now.~~

Picking left ones, following meeting.

Vianvolaeus commented 3 years ago

Facial reshaping following the simplification input. Resized iris, reshaped eyes, etc.

Will require a lot of redoing on shapekeys, but was able to get some rough previews out with some mixing.

Not going to replace the working file just yet since there's still a bunch of work to be done rebuilding facial shapekeys, etc.

avaer commented 3 years ago

Those eyes irises are too big for that eye shape, and too dark. gear.vrm is still the best thing we have.

avaer commented 3 years ago

I would also copy the gear.vrm hair much more closely, since that is what the 2D renditions use.

Vianvolaeus commented 3 years ago

Tried to hit a midpoint between the proposed 2d art and the gear 3d model, and retain some of the shapes that are expresed at different angles in the 2D. They do change a bit between art / angles, but I've tried to just capture an 'overall' vibe of it..

Outline shading seems to help a lot, and I've also set up the UV space so that the inside and minor edge of the hair will be in darker shadow.

Front of the hair is following the gear and 2D more - shaped up the rear of the hair to be more like the 2d rather than the messy long Gear 3D version,

Overall it is using a lot less individual shapes and rigged parts, so it should move more uniform and with less collisions, etc.

Face is also a midpoint between the two, since gear was supposed to be close, but the 2d is different in eye shape / head shape as well - adjustments made all round.

Some texture work - new iris tes, new eyelashes to fit 2d art, some fake AO / shadows in some aspects to decrease the flat feel it was having.

If this is a-ok, it's ready to be transplanted onto the existing body/clothing and begin the VRM process / optim. etc.

There's a VRM model (before adding clothing etc), and a .blend file on the drive if you wish to take a look at it in more detail:

https://drive.google.com/drive/folders/1hWy0YQUwFH_AcUfvH1WVis26bePcF0WZ

avaer commented 3 years ago

This is real good, looks about as close as we can get to 2D in the 3D. Seems ready to be glued to a full body.

I tried the vrm in engine and it looks great; it also looks like hair rigs would take it to the next level in motion, which was the whole point of the hair.

avaer commented 3 years ago

For the body, I’d again defer to the 2d art, which is relatively close to what we already had (the goal being easy to draw, and having parity between mediums).

That includes the green hue, long socks and possibly simpler shoes (the current ones are kind of “turbo” in a good way, but they are harder to draw). One interpretation of that sketch could be Zelda/Link-style boots, which are very easy to draw and seem to fit the character well too.

webaverse / app