search

ali-vilab / composer

Official implementation of "Composer: Creative and Controllable Image Synthesis with Composable Conditions"
MIT License
1.53k stars 48 forks source link

When is the training and inference code going to be released? #2

Open geekyayush opened 1 year ago

geekyayush commented 1 year ago

Any idea, when the training and inference code going to be released?

This looks exciting

dawei03896 commented 1 year ago

+1

hyd998877 commented 1 year ago

+1

sntlks commented 1 year ago

When?

KleinXin commented 1 year ago

looking forward

alelordelo commented 1 year ago

+1

Rage997 commented 1 year ago

Probably never - that's how these companies like to play the game. They create hype about their researchers but 90% of the times they never release any source code, and when they do, it's spaghetti/research code

ghpkishore commented 1 year ago

HI @Rage997 I do not think that is the case as mentioned in this issue. Mid March is the likely date as of now. https://github.com/modelscope/modelscope/issues/160

Rage997 commented 1 year ago

@ghpkishore hopefully. I am looking forward to play with the source code!

huanglianghua commented 1 year ago

Hi, thank you for your interest in Composer! We're currently tidying up our code and going through some necessary internal processes. Once that's done, we will release the code and models. Thanks.

aartykov commented 1 year ago

Hey, @huanglianghua congrats on the great work. Currently, I am trying to implement the paper onto the stable diffusion. My question is, what does the following sentence mean; "Project image embeddings and color palettes into eight extra tokens and concatenate them with CLIP word embeddings"? So what are these eight extra tokens?

Thanks in regard, Arslan

huanglianghua commented 1 year ago

Hi, @artykov1511 awesome! Looking forward to your sd version composer!

Assuming a batch size of B, the shapes of color palettes, image embeddings, and word embeddings are (Bx156), (Bx768), and (Bx77x768), respectively. The steps are:

  1. We use "fc->silu->fc" layers to project color palettes to a shape of (Bx2048) and reshape it to (Bx4x512).
  2. We use another stack of "fc->silu->fc" layers to project image embeddings to a shape of (Bx2048) and reshape it to (Bx4x512).
  3. We project word embeddings to a shape of (Bx77x512) using another stack of "fc->silu->fc" layers.
  4. We concatenate the three tensors along the second dimension, resulting in a tensor of shape (Bx85x512), which is then used as the context and fed to UNet's cross-attention modules.
aartykov commented 1 year ago

Hey @huanglianghua! Thanks for your feedback! And I also similarly projected text embeddings to a shape of (B, 77768) and added it to timestep embedding of shape (B, 4 self.model_channels) along with img and color embeddings of shape (B, 768), (B, 156), respectively. Did I implement it correctly?

ethansmith2000 commented 1 year ago

hey @artykov1511 see UnCLIPPipeline in diffusers which uses the same methodology of projection onto timestep embeddings and extra context tokens :)

ghpkishore commented 1 year ago

Hi @huanglianghua , the entire community is eagerly waiting for you to release the code. Please let us know when it would be possible. Checking 5 times a day everyday now for code release. 🙏

phongnhhn92 commented 1 year ago

Hey guys, can we stop asking the team to release the code. I think they know we are all waiting but we need to give them some time to polish it.

huanglianghua commented 1 year ago

Dear everyone,

I wanted to take a moment to provide an update on our plans for Composer's open source release. We have been working hard to prepare for this release, but unfortunately, we have had to postpone it until April. This delay is due to internal audits and policy requirements related to AIGC, and we are not allowed to disclose the exact date. However, we want to assure you that we are still working diligently to make sure that the final release is of the highest quality.

In addition to the delay, we are excited to announce that the 5B-Dalle2-like model will also be released alongside Composer in April. We believe that this model will be a valuable addition to our open source offerings and we can't wait to share it with you.

As we continue to fine-tune the final performance of Composer in the coming days, we appreciate everyone's enthusiasm and support. If you have any questions or concerns, please do not hesitate to reach out to me or the team.

Best regards

aartykov commented 1 year ago

Dear all,

I am excited to announce that I have begun implementing a lightweight Latent Composer based on the stable diffusion model. I have already completed the majority of the implementation and am currently working on the remaining tasks outlined in the repository.

If you are interested in contributing to this project, please feel free to submit a pull request and I will review your code promptly. Your contributions are greatly appreciated and if you find my work useful, please consider giving the repository a star.

If you have any questions or concerns, please do not hesitate to open an issue on the repository. I am happy to address any inquiries you may have.

The code: https://github.com/aartykov/Latent-Composer-pytorch

Best regards, Arslan

Ukuer commented 1 year ago

Great work @aartykov

ninjasaid2k commented 1 year ago

Dear everyone,

I wanted to take a moment to provide an update on our plans for Composer's open source release. We have been working hard to prepare for this release, but unfortunately, we have had to postpone it until April. This delay is due to internal audits and policy requirements related to AIGC, and we are not allowed to disclose the exact date. However, we want to assure you that we are still working diligently to make sure that the final release is of the highest quality.

In addition to the delay, we are excited to announce that the 5B-Dalle2-like model will also be released alongside Composer in April. We believe that this model will be a valuable addition to our open source offerings and we can't wait to share it with you.

As we continue to fine-tune the final performance of Composer in the coming days, we appreciate everyone's enthusiasm and support. If you have any questions or concerns, please do not hesitate to reach out to me or the team.

Best regards

Hi, any update about code?

LangDaoAI commented 1 year ago

Continue to postpone when?

geonm commented 1 year ago

Try this before Composer comes out. It's Graphit we released and you might find something interesting. I also hope Composer will be released soon.

https://github.com/navervision/Graphit

aartykov commented 1 year ago

Dear everyone, I wanted to take a moment to provide an update on our plans for Composer's open source release. We have been working hard to prepare for this release, but unfortunately, we have had to postpone it until April. This delay is due to internal audits and policy requirements related to AIGC, and we are not allowed to disclose the exact date. However, we want to assure you that we are still working diligently to make sure that the final release is of the highest quality. In addition to the delay, we are excited to announce that the 5B-Dalle2-like model will also be released alongside Composer in April. We believe that this model will be a valuable addition to our open source offerings and we can't wait to share it with you. As we continue to fine-tune the final performance of Composer in the coming days, we appreciate everyone's enthusiasm and support. If you have any questions or concerns, please do not hesitate to reach out to me or the team. Best regards

Hi, any update about code?

Hi! I am still debugging the code. Since I can focus on the project only in my spare time, the debugging process goes slowly. I will let you know if there is any update. Thanks for your patience.

geekyayush commented 1 year ago

Dear everyone, I wanted to take a moment to provide an update on our plans for Composer's open source release. We have been working hard to prepare for this release, but unfortunately, we have had to postpone it until April. This delay is due to internal audits and policy requirements related to AIGC, and we are not allowed to disclose the exact date. However, we want to assure you that we are still working diligently to make sure that the final release is of the highest quality. In addition to the delay, we are excited to announce that the 5B-Dalle2-like model will also be released alongside Composer in April. We believe that this model will be a valuable addition to our open source offerings and we can't wait to share it with you. As we continue to fine-tune the final performance of Composer in the coming days, we appreciate everyone's enthusiasm and support. If you have any questions or concerns, please do not hesitate to reach out to me or the team. Best regards

Hi, any update about code?

Hi! I am still debugging the code. Since I can focus on the project only in my spare time, the debugging process goes slowly. I will let you know if there is any update. Thanks for your patience.

Hey, Do let us know if we can help.

I think we all would love to do that.

aartykov commented 1 year ago

Dear everyone, I wanted to take a moment to provide an update on our plans for Composer's open source release. We have been working hard to prepare for this release, but unfortunately, we have had to postpone it until April. This delay is due to internal audits and policy requirements related to AIGC, and we are not allowed to disclose the exact date. However, we want to assure you that we are still working diligently to make sure that the final release is of the highest quality. In addition to the delay, we are excited to announce that the 5B-Dalle2-like model will also be released alongside Composer in April. We believe that this model will be a valuable addition to our open source offerings and we can't wait to share it with you. As we continue to fine-tune the final performance of Composer in the coming days, we appreciate everyone's enthusiasm and support. If you have any questions or concerns, please do not hesitate to reach out to me or the team. Best regards

Hi, any update about code?

Hi! I am still debugging the code. Since I can focus on the project only in my spare time, the debugging process goes slowly. I will let you know if there is any update. Thanks for your patience.

Hey, Do let us know if we can help.

I think we all would love to do that.

Hey, actually would be glad if someone gives a hint about where the dropout operation on conditionings is implemented during training in stable diffusion.

loboere commented 1 year ago

Now can you give us a date when it will be released?

AugustRush commented 1 year ago

鸽了鸽了😂

aartykov commented 1 year ago

Dear all,

I am excited to announce that I have begun implementing a lightweight Latent Composer based on the stable diffusion model. I have already completed the majority of the implementation and am currently working on the remaining tasks outlined in the repository.

If you are interested in contributing to this project, please feel free to submit a pull request and I will review your code promptly. Your contributions are greatly appreciated and if you find my work useful, please consider giving the repository a star.

If you have any questions or concerns, please do not hesitate to open an issue on the repository. I am happy to address any inquiries you may have.

The code: https://github.com/aartykov/Latent-Composer-pytorch

Best regards, Arslan

Dear Community! I debugged and updated the repo. So, now, you should be able to train the Latent Composer Network without errors. For any questions, please, open an issue on the repo.

Best, Arslan

josephrocca commented 1 year ago

https://github.com/modelscope/modelscope/issues/160#issuecomment-1535631391

Due to internal audit requirements, the open-sourcing of the Composer has been put on hold indefinitely. We are still working with the Composer team towards a final resolution for sharing it with the community. Unfortunately, there is simply no definite timeline at this moment.

:(

danigoju commented 1 year ago

Damn @Rage997 was right all this time. We might have been bamboozled

Amazingldl commented 1 year ago

It is July....

Damn @Rage997 was right all this time. We might have been bamboozled

geekyayush commented 1 year ago

Probably never

I guess you were right

JaosonMa commented 11 months ago

just wait!

OrangeSodahub commented 8 months ago

just wait!

geekyayush commented 8 months ago

Feels like a decade has passed and I am starting to forget why I asked this question in the first place. So I don't know what to wait for anymore.

ninjasaid2k commented 8 months ago

Feels like a decade has passed and I am starting to forget why I asked this question in the first place. So I don't know what to wait for anymore and doesn't need 40GB GPUs.

At this point everything composer can do can be replicated with Ipadapter or controlnet or some other model/adapter.