search

Unity-Technologies / com.unity.cv.synthetichumans

A package for creating Unity Perception compatible synthetic people.
Other
102 stars 20 forks source link

How to render depth map of human bodies (without clothes)? #3

Closed davidfant closed 1 year ago

davidfant commented 1 year ago

I'm using Unity Perception and Synthetic Humans to generate renders of humans. Using the depth labeler I can get a depth map of the actual human being rendered.

I want to generate the depth map from the raw human body, excluding the clothes. How would I do that? I assume I'd have to modify the DepthChannel to exclude the clothes somehow? https://github.com/Unity-Technologies/com.unity.perception/blob/f45895f7dcad27dee545d6165a2f6c237554600a/com.unity.perception/Runtime/GroundTruth/Sensors/Channels/DepthChannel.cs

csun commented 1 year ago

Just to clarify, you want the clothes to render in the normal (color) image, but want them to be ignored in the depth image?

From the other issue you linked, it looks like you can achieve this by setting the clothing objects to a different layer and then making it so that the depth labeler ignores that specific layer. I don't know off the top of my head how to do the second part of that (sounds like it's implemented as a feature on the perception camera), but here are some tips on doing the first part:

  1. Create a subclass of the HumanGenerationLifecycleSubscriber class and add it to your base prefab (look for the mention of base prefabs here).
  2. Override the OnGenerationComplete method in your lifecycle subscriber and use gameObject.GetComponent<SingleHumanGenerationAssetRefs>().createdClothing to get access to the generated clothing items.
  3. Iterate through those clothing items and set their layer.

With all of that done, you should be able to make it so that you can ignore them with certain labelers. You may need to play around with the perception camera yourself to figure this bit out, as I've never actually done it