Project Grease Pencil into mesh

[Dereuz24]

Hi there!

I was wondering if maybe you have thought on a way to draw straight into mesh from a camera projection without masking by transforming a mesh into grease pencil?

This kind of stuff would allow the built interpolation or extrapolation functionality to follow a mesh across the timeline in a more convenient manner. Currently the solutions are a bit cumbersome having to convert the mesh to GP before masking and using the placing stroke on surface afterward. Basically everything is already built into blender but there's a lot of steps and adjustments in between that could otherwise be very straightforward

If this could be added as a more user friendly function it would make things a lot more easier and satisfying to work with

EDIT: A script like solution I thought about for an animated object is something along the lines of

1. User draws stroke
2. Click on button that basically runs: Clean-up project into surface from camera view
3. User draws next keyframe
4. Click on project into surface
5. Then click on complete and this runs a series of steps like:
6. Going to Frame in the middle of these two strokes.
7. Interpolate with selected interpolation method
8. Clean-up project into surface
9. Repeat until desired number of frames is completed

As for the masking automatation, maybe the selected mesh is selected before the last operation and converted into a clean closed gpen sequence with a solid color (ignoring all materials), preserving the original mesh and automatically placed in a hidden collection, and this new Gpen sequence is placed in the masks menu.

Something along those lines more or less.

[chsh2]

Hi. This seems to be a workflow that I am not quite familiar with, since this add-on mainly focuses on drawing on a plane. I occasionally draw on the surface of meshes, but I do not know what the masking refers to in your description. Are there any tutorials or demos showing how the masking works?

The procedure you mentioned also reminds me of some Blender functions, but I am not sure if they are related:

• There is an option Stroke Placement to snap newly drawn Grease Pencil strokes to the mesh surface.
• There is an operator Bake Mesh to Grease Pencil that works for multiple frames.

[Dereuz24]

Hi there. Sorry for the delay

Yes absolutely. So I just meant a typical clipping mask to paint on an area of a defined layer like in 2D drawing software but with the grease pencil:

• https://www.youtube.com/watch?v=cw5C7AjvB1U

That's a good example of it. He's just painting a shadow only in the area defined as skin.

And absolutely! What I mentioned I'm currently doing it with some macros and scripts because blender does has those operators individually.

So the idea with this is that because NPR work in Blender is so limited, even with custom render engines like malt, sometimes the only solution you have at your disposal is to hand drawn the shadows to not have to deal with problematic lighting, normals or meshes. Sometimes is just faster to use a broken poorly planned topology and just paint or correct shadows on top of it.

I've done that plenty on 2D drawing software but blender does offer decent interpolation for the grease pencil so the idea in general is to use the stroke placement on surface from camera position, and use the selected mesh itself as the mask like in that 2D example.

The stroke placement works very well. Let's say you have a character on frame, and you want to draw an slit on their cheek, you can do it with this method, project the grease pencil onto the surface, then jump to the last frame of the shot, draw the second frame, and then slowly interpolate in-betweens and fix.

The complex part is when you want to start painting shadows. Because they have to be contained by the character/object.

It's been a while since I've used the vanilla version of "bake mesh to grease pencil" because back then it left a lot to be desired, edges where split, sometimes weird edges got converted to grease pencil strokes themselves, so a lot had to be cleaned up.

That's why now I use addons like Sketchn'carve because it offers a lot of control and refinement in the baking process.

Once the mesh is converted to grease pencil I can just select that as a mask and paint my shadows freely. Then use the same method, go to the last frame, draw again, then go to the middle, interpolate, then go to cleanup "Reproject strokes" and getting them to snap from camera view into the surface. This does better than snapping sometimes with each vertex sticking to the surface

After these 3 frames are secured then routinely I just run the same interpolation in the in-betweens with reprojection until I have all frames.

There's a lot of steps and a lot of poor functionality. Meshes converted to grease pencil tend to be very heavy so if we were to have operators to simply define the mesh, do our painting at the beginning, end frame, then fix the middle if the user choses, and after that just run the whole thing in 1's or 2's or 3's and whatever interval we pick, then ditching the grease pencil layer used for masking after all is completed, there could be a more usable way to do all of this like it's 2D software. I was about to just try and put together that stuff into an addon with my limited knowledge but given your addon is very polished for a lot of the operations I thought that maybe you have good ideas for cleaner more efficient mesh conversion and how to make this stuff into an usable menu without all the clunk

I don't know if it all makes sense

EDIT: Also, I really don't know how to use those masks and then automatically delete everything drawn outside of the masked area from camera view. If I was to do all that stuff automatically then there needs to be a procedure to clean it all before deleting the grease pencil conversion. So little details like that keep me just using a dumb step by step method