How to use "Cross-Image region drag and merge"?

[hzhou17]

I am sorry if this is not the place to ask questions.

I launched the gradio through editany.py. But I did not figure out how to select a green region and drag merge another image, as shown in the Features area. I think it is really cool to drag and merge the outfit of the superman.

Would really appreciate it if the author or anybody else could help me out.

[gasvn]

To reproduce our results, you can launch the editany_test.py And there is a reference tab in the grdio demo. You can update the image in reference tab then select the region you want to drag. We will update the readme file, thanks.

[Guanyuansheng]

To reproduce our results, you can launch the editany_test.py And there is a reference tab in the grdio demo. You can update the image in reference tab then select the region you want to drag. We will update the readme file, thanks.

The environment required by editany_test.py is not consistent with environment.yaml (for example, running editany_test.py with the environment installed by environment.yaml will report multiple errors such as such as missing xformers and diffusers versions), please update the readme file as soon as possible. Thanks for your contribution!

[ennnnny]

Curious about the effect of "Cross-Image region drag and merge". I tried to run it in google colab and found that the memory exceeded 12G causing the startup to fail, and there may be a problem with environment.yaml. It would be great if could provide a google colab configuration!😀

[gasvn]

To reproduce our results, you can launch the editany_test.py And there is a reference tab in the grdio demo. You can update the image in reference tab then select the region you want to drag. We will update the readme file, thanks.

The environment required by editany_test.py is not consistent with environment.yaml (for example, running editany_test.py with the environment installed by environment.yaml will report multiple errors such as such as missing xformers and diffusers versions), please update the readme file as soon as possible. Thanks for your contribution!

Thanks for the feedback, I have updated the packages in environment.yaml. Please let me know if you still encounter errors.

[gasvn]

Curious about the effect of "Cross-Image region drag and merge". I tried to run it in google colab and found that the memory exceeded 12G causing the startup to fail, and there may be a problem with environment.yaml. It would be great if could provide a google colab configuration!😀

Other extra modules like BLIP model and SAM model may cost some GPU memory. You can extract ths function https://github.com/sail-sg/EditAnything/blob/572298811b13d90569fb39bbf48e624168e9a005/editany_lora.py#L825 to use the cross-image drag in the google colab. This would aovid the OOM problem I suppose.

[gamingflexer]

Curious about the effect of "Cross-Image region drag and merge". I tried to run it in google colab and found that the memory exceeded 12G causing the startup to fail, and there may be a problem with environment.yaml. It would be great if could provide a google colab configuration!😀

Other extra modules like BLIP model and SAM model may cost some GPU memory. You can extract ths function

https://github.com/sail-sg/EditAnything/blob/572298811b13d90569fb39bbf48e624168e9a005/editany_lora.py#L825

to use the cross-image drag in the google colab. This would aovid the OOM problem I suppose.

I am also getting issues. I have attached the collab here. Really appreciate your help !

[ennnnny]

Curious about the effect of "Cross-Image region drag and merge". I tried to run it in google colab and found that the memory exceeded 12G causing the startup to fail, and there may be a problem with environment.yaml. It would be great if could provide a google colab configuration!😀

Other extra modules like BLIP model and SAM model may cost some GPU memory. You can extract ths function

https://github.com/sail-sg/EditAnything/blob/572298811b13d90569fb39bbf48e624168e9a005/editany_lora.py#L825

to use the cross-image drag in the google colab. This would aovid the OOM problem I suppose.

environment.yaml need safetensors>=0.3.1

Thanks for the update! I tested it on a 3080Ti graphics card with 32G RAM machine and was able to run it, but the generated graphs were very resource intensive. Tried several times and failed to achieve similar results as the demo. Hopefully a more detailed tutorial on how to do this will follow.

[ennnnny]

Curious about the effect of "Cross-Image region drag and merge". I tried to run it in google colab and found that the memory exceeded 12G causing the startup to fail, and there may be a problem with environment.yaml. It would be great if could provide a google colab configuration!😀

Other extra modules like BLIP model and SAM model may cost some GPU memory. You can extract ths function https://github.com/sail-sg/EditAnything/blob/572298811b13d90569fb39bbf48e624168e9a005/editany_lora.py#L825

to use the cross-image drag in the google colab. This would aovid the OOM problem I suppose.

I am also getting issues. I have attached the collab here. Really appreciate your help !

You can try my https://github.com/ennnnny/sd_colab/blob/self/editanything.ipynb but I haven't solved the OOM problem. Maybe the colab Pro account can perform.

[gamingflexer]

Curious about the effect of "Cross-Image region drag and merge". I tried to run it in google colab and found that the memory exceeded 12G causing the startup to fail, and there may be a problem with environment.yaml. It would be great if could provide a google colab configuration!😀

Other extra modules like BLIP model and SAM model may cost some GPU memory. You can extract ths function https://github.com/sail-sg/EditAnything/blob/572298811b13d90569fb39bbf48e624168e9a005/editany_lora.py#L825

to use the cross-image drag in the google colab. This would aovid the OOM problem I suppose.

I am also getting issues. I have attached the collab here. Really appreciate your help !

You can try my https://github.com/ennnnny/sd_colab/blob/self/editanything.ipynb but I haven't solved the OOM problem. Maybe the colab Pro account can perform.

Thanks will look into it !

[gasvn]

https://github.com/ennnnny/sd_colab/blob/self/editanything.ipynb

Curious about the effect of "Cross-Image region drag and merge". I tried to run it in google colab and found that the memory exceeded 12G causing the startup to fail, and there may be a problem with environment.yaml. It would be great if could provide a google colab configuration!😀

Other extra modules like BLIP model and SAM model may cost some GPU memory. You can extract ths function https://github.com/sail-sg/EditAnything/blob/572298811b13d90569fb39bbf48e624168e9a005/editany_lora.py#L825

to use the cross-image drag in the google colab. This would aovid the OOM problem I suppose.

environment.yaml need safetensors>=0.3.1

Thanks for the update! I tested it on a 3080Ti graphics card with 32G RAM machine and was able to run it, but the generated graphs were very resource intensive. Tried several times and failed to achieve similar results as the demo. Hopefully a more detailed tutorial on how to do this will follow.

As this solution is training-free, you need to adjust the parameters to get the good results. Also, I find that the text prompt is important. If you cannot get a good description of your reference region, you can train the reference region with text inversion to get a good text embedding. I will upload a tutorial, thanks for your advice.

[gamingflexer]

https://github.com/ennnnny/sd_colab/blob/self/editanything.ipynb

Curious about the effect of "Cross-Image region drag and merge". I tried to run it in google colab and found that the memory exceeded 12G causing the startup to fail, and there may be a problem with environment.yaml. It would be great if could provide a google colab configuration!😀

Other extra modules like BLIP model and SAM model may cost some GPU memory. You can extract ths function https://github.com/sail-sg/EditAnything/blob/572298811b13d90569fb39bbf48e624168e9a005/editany_lora.py#L825

to use the cross-image drag in the google colab. This would aovid the OOM problem I suppose.

environment.yaml need safetensors>=0.3.1 Thanks for the update! I tested it on a 3080Ti graphics card with 32G RAM machine and was able to run it, but the generated graphs were very resource intensive. Tried several times and failed to achieve similar results as the demo. Hopefully a more detailed tutorial on how to do this will follow.

As this solution is training-free, you need to adjust the parameters to get the good results. Also, I find that the text prompt is important. If you cannot get a good description of your reference region, you can train the reference region with text inversion to get a good text embedding. I will upload a tutorial, thanks for your advice.

Results were not that great but hopefully your tutorial can help us !

Tutorial for this would be great. Thanks waiting for it : )

[ZhouYaoxue]

the "Cross-image region drag and merge" is great, but which files can i read to know how it works

[schananas]

did someone managed to make tutorial for cross image region drag and merge? really curious to try out this functionality

[zhengyi0533]

How to use it with controlnet?