search

TencentARC / InstantMesh

InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Models
Apache License 2.0
3.06k stars 322 forks source link

Automatic1111 compatibility #28

Closed safouene-jebali closed 5 months ago

safouene-jebali commented 5 months ago

Hello is it possible to add InstantMesh as extension in Automatic1111 WebUi ?

KhalilSRIDI commented 5 months ago

+1

jtydhr88 commented 5 months ago

Yeah, it is possible, I implemented the similar thing in https://github.com/jtydhr88/sd-webui-txt-img-to-3d-model by OpenAI Sharp-E. However the difficulty of InstantMesh is that it depends some dependencies may be challenging for user to install. While I implemented ComfyUI node https://github.com/jtydhr88/ComfyUI-InstantMesh, I choose a simple solution that asks user to install https://github.com/MrForExample/ComfyUI-3D-Pack first, which has a better solution to install the dependencies.

in fact, I don’t see it is high value to integrate InstantMesh into sd-webui, if you are able to run it on your local, you could easily use it to process the image generated from sd-webui.

however, if I could spare some time on weekend, I may take a look at it

safouene-jebali commented 5 months ago

That sounds great! I really appreciate the work you've done with ComfyUI. In fact, I'm currently working on a project where all demos must be compatible with automatic1111, which is why I was keen on implementing InstantMesh directly in the sd-webui. I look forward to seeing any updates you might explore over the weekend regarding this integration. Thanks again for considering it!

jtydhr88 commented 5 months ago

https://github.com/jtydhr88/sd-webui-InstantMesh I implemented this initial version, I will update readme for setup later image

DiamondGlassDrill commented 5 months ago

Awesome :) thanks!!!!

DiamondGlassDrill commented 5 months ago

So with txt2image i can push over as input for InstantMesh Extension wrapper?

jtydhr88 commented 5 months ago

Not now, current version is just an initial version, I just moved the gradio ui of this repo into sd-webui, without any new features. Btw, I also need to mention, to make it working on sd-webui requires some additional steps which I don’t finish the readme, if you want to try at this moment, you could refer to my personal note:

  1. conda create --name sd-webui-19 python=3.10
  2. conda deactivate
  3. pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
  4. pip install -r requirements.txt
  5. conda install -c conda-forge cudatoolkit-dev -y
  6. conda install Ninja
  7. pip install https://huggingface.co/r4ziel/xformers_pre_built/resolve/main/triton-2.0.0-cp310-cp310-win_amd64.whl
  8. pip install imageio[ffmpeg]
  9. cd extensions\sd-webui-InstantMesh
  10. pip install -r requirements.txt
  11. cd ....
  12. python launch.py --disable-safe-unpickle
safouene-jebali commented 5 months ago

image_2024-04-21_153253480 It worked perfectly for me following the recommended steps !