search

gnobitab / InstaFlow

:zap: InstaFlow! One-Step Stable Diffusion with Rectified Flow (ICLR 2024)
MIT License
1.14k stars 36 forks source link

I think the author is more likely not to release the training code and pre trained models #8

Closed YangPanHZAU closed 9 months ago

YangPanHZAU commented 1 year ago

The reasons are as follows: I am a worker in the field of the acceleration inference step in text-to-image diffuison, the similarity works such as Snapfusion and W-condition model are not open source currently. Snapfusion: Text-to-Image Diffusion Model on Mobile Devices within Two Seconds,https://github.com/snap-research/SnapFusion W-condition: On Distillation of Guided Diffusion Models (CVPR 2023, Award candidate).

If the author does not intend to open source the code, I will reproduce the InstaFlow. Could you please communicate with me about the InstaFlow.

pietrobolcato commented 1 year ago

Please do! An opensource implementation would be awesome, let us know if we can help!

spacewalkingninja commented 1 year ago

Yes this is very interesting! Thanks for contributing!

putuoka commented 12 months ago

Don't open source it yet. First create a commercial version that can be sold to companies. After that, you can work on an open source version separately, similar to how StableAudio did their release.

spacewalkingninja commented 11 months ago

hello any news, where is the model?

spacewalkingninja commented 11 months ago

Don't open source it yet. First create a commercial version that can be sold to companies. After that, you can work on an open source version separately, similar to how StableAudio did their release.

THIS IS REQUIREMENT BY LAW: OPEN SOURCE MUST BE RELEASED FIRST ASAP POOR PEOPLE NEED IT AROUND THE WORLD!!!! POOR PEOPLE CANNOT AFFORD SOFTWARE & HIGH END HARDWARE!!!! THINK OF THEIR NEEDS FIRST BEFORE MULTIMILLION COMPANIES!!!

WyattAutomation commented 11 months ago

Yall.. The paper appears to have enough information that even if they don't release it, I would bet money Lucidrains or the OP of this issue will pull through and release a community version.

YangPanHZAU above mentions that "if they don't release it" they are capable of reproducing the paper. Someone did this for DragGAN and they eventually released that one; think about it for a second--do you want to spend all that effort just to flex your ability to turn a whitepaper into code if they release the full model?

All we need from the Authors is a clear and honest "yes" or "no"; if it is "no" then building a community version makes sense and is well within the ability of at least a couple people in this thread.

Patience is valuable. With all that said though, I have a proposition to suggest:

Perhaps we set ourselves a "cutoff date", where after that point, if the author has not answered if/when they will release the code/models, we stop waiting and begin work on an Open Source version?

I feel like this should become a more common pattern:

1) Set an agreed upon date amongst potential contributors to start working on a community version, if the author has not updated us on the status of code/model sharing by then

2) Communicate that date to the author of the repo, and let them know that it would be kind of them to give an honest "yes" or "no" on or before that date, or that they can let us know if we should give them more time. If they need more time, we adjust the start-date of community efforts as needed.

The author isn't obligated to share anything nor communicate with any of us. I want access to this code/models too but we can't depend entirely on handouts from funded entities if we want a robust Open Source community. If you are truly passionate about Democratizing technology, let's express that passion through organization and contribution.

Don't discount the communities ability to provide a solution, and don't disrespect the author for being bound to rules they may not even have any control over.

Pevernow commented 11 months ago

@gnobitab If there is still no reply within the next week, we will assume that you are unwilling to provide models and source code, and try to start forging the community version.

Pevernow commented 11 months ago

@YangPanHZAU My abilities are limited, but I would love to write webui for Instaflow.

danieltanhx commented 11 months ago

https://github.com/huggingface/diffusers/issues/5256

oxysoft commented 11 months ago

This is incredibly important work, with 1 step diffusion we get as many FPS as we get it/s, so we are entering the realm of realtime.

gnobitab commented 9 months ago

Code and pre-trained models are released.

spacewalkingninja commented 9 months ago

Code and pre-trained models are released.

NO THEY ARE NOT!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! IT IS NOT OPEN SOURCE!!!!!!!!!!!!! OPEN SOURCE COMMUNITY BETRAYAL!!!!!!!!!!!!!!!!!!!!

oxysoft commented 9 months ago

Lol chill, it's right there

Huge thanks to the author for coming thru, this is gonna be a blast.

WyattAutomation commented 2 hours ago

I just got ~21 FPS realtime SD + MultiControlNet working in my app's pipeline because of this. InstaFlow + DMD = Cookin' with gas.

https://github.com/user-attachments/assets/239afba6-9a35-4bcc-9a6f-928e92bc8fd1

There are many other optimizations in my pipeline but this was the remaining piece to get it going this fast.

I have this working with LayerDiffusion as well -- that means I should be able to spin up multiple pipelines now and do realtime composite image generation. Load up a specialized pipeline for any stream of ControlNet elements you want from your game engine of choice.

Next up: going to set up an LLM agent with function calling in Unity to start dropping-in OpenPose enemies and LooseControl cubes.. Fully realtime AI rendered videogame anyone?

It's getting there, thank you SO MUCH for releasing this!!