search

zqh0253 / 3DitScene

3DitScene: Editing Any Scene via Language-guided Disentangled Gaussian Splatting
https://zqh0253.github.io/3DitScene/
159 stars 8 forks source link

When I want to delete this bear in the picture, it failed. #11

Closed ZhaoLongjiea closed 3 days ago

ZhaoLongjiea commented 1 month ago

When I want to test the inpainting ability, and I revised the prompt as 'delete the bear'. Nothing happened. I still got a photo with this bear. And after I check the scene_lang.yaml. How to modificate the command line? Below is the command and photo after I used 'delete the bear'.

python -u launch.py --config custom/threestudio-3dgs/configs/scene_lang.yaml --train --gpu 0 tag=3DitScene "system.geometry.geometry_convert_from=depth:assets/teddy.png" "system.geometry.ooi_bbox=[122,119,387,495]" "system.prompt_processor.prompt=delete the bear" "system.empty_prompt=Times Square, out of focus" "system.side_prompt=Times Square, out of focus"

outpaint_0

zqh0253 commented 1 month ago

The prompt should be used to describe the image itself, not the editing purpose. To edit the image, such as removing the foreground bear, you need to manipulate the 3D Gaussians associated with the object.

In GaussianBaseModel, ooi_masks_0 provides the mask for the foreground object, while _opacity_mask defines the opacity of each Gaussian. To remove the object, you can set the opacity of foreground object to zero.