Possible to use model texture data as particle color?

keijiro / Smrvfx

Skinned mesh sampling with VFX Graph

The Unlicense

1.08k stars 153 forks source link

Possible to use model texture data as particle color? #16

Closed mellinger closed 3 years ago

mellinger commented 4 years ago

I was wondering if it was possible to use texture data from the model to color the particles?

I'm looking to make something similar to your Depthkit/Realsense examples (https://camo.githubusercontent.com/2bc3c65923f8e3db7131ab50e1254164ac25c558/68747470733a2f2f692e696d6775722e636f6d2f4b304338304c662e676966), a kind of enhanced point cloud effect.

Any tips are appreciated. Thanks!

keijiro commented 4 years ago

There are two possibilities:

Bake the UVs into a map in the same way to position/normal maps. Then you can sample the texture in a VFX based on UVs retrieved from the UV map.
Bake the texture into vertex colors using an external DCC tool (Houdini, etc.) Bake the vertex colors into a color map, then you can use it in a VFX in the same way to colored point clouds like shown in RealSense/Kinect examples.

The latter solution is faster (in performance-wise) and more straightforward, but it requires preprocessing with an external tool.

mellinger commented 4 years ago

Great, I’ll see if I can figure out a workflow from that. Please let me know if you have any similar examples to draw from! Thanks!

liudixian commented 3 years ago

There are two possibilities:

Bake the UVs into a map in the same way to position/normal maps. Then you can sample the texture in a VFX based on UVs retrieved from the UV map.

Bake the texture into vertex colors using an external DCC tool (Houdini, etc.) Bake the vertex colors into a color map, then you can use it in a VFX in the same way to colored point clouds like shown in RealSense/Kinect examples.

The latter solution is faster (in performance-wise) and more straightforward, but it requires preprocessing with an external tool.

Hello, Smrvfx is great. And I have the same demand. Now I have try the second way to bake a vertex color to pointcache. But I find the uv is incorrect. Is there any solution?

keijiro commented 3 years ago

@liudixian I don't know how you baked the vertex colors, so I have no idea about the problem.

liudixian commented 3 years ago

shoot01

Oh, I can provide more detailed information. I imported the model's fbx and materials into houdini, and then exported the pointcache with color information using the official houdini export pointcache example.

keijiro commented 3 years ago

@liudixian

Thanks for the explanation. There seems to be a misconception about what I told in the original post.

What I was trying to say is that you can bake sampled texture colors into vertex color attributes. In other words, you can bake texture colors into the "Cd" attribute on Houdini and export it as a fbx file that has vertex color attributes.

Moreover, you have to extend Smrvfx to read those vertex colors (the current implementation doesn't support vertex colors). So, this is not an easy way to solve the problem.

liudixian commented 3 years ago

Oh, now I got it. It really not an easy way for me. But thanks. : )

laumagg commented 3 years ago

Hi! First of all, thank you for this amazing tool! I am also trying to use texture data for the color. Following this tutorial it worked, but it overrides the Set Position From Map block from the SMRVFX, so my mesh isn't moving anymore. How can I do this? I can't understand how to connect the position from the point cache without loosing the movement. Here is my Graph, without the position from the SMRVFX. I thank you very much in advance!

Screenshot (257)

keijiro commented 3 years ago

Now VFX Graph officially supports Skinned Mesh Renderer sampling. You can easily sample the texture using the texture coordinates from the skinned mesh.

https://docs.unity3d.com/Packages/com.unity.visualeffectgraph@12.0/manual/whats-new-11.html

I close this issue as fixed in the official implementation.