search

huggingface / diffusers

🤗 Diffusers: State-of-the-art diffusion models for image and audio generation in PyTorch and FLAX.
https://huggingface.co/docs/diffusers
Apache License 2.0
25.03k stars 5.17k forks source link

[Docs] Doc Sprint on Wednesday Sep 7 #293

Open patrickvonplaten opened 2 years ago

patrickvonplaten commented 2 years ago

There is still more TODO, to see open options, see this comment

Let's add them docs.

cc @anton-l

patrickvonplaten commented 2 years ago

Docs are now live :tada:

Thanks to @natolambert we have a rough template of diffusers docs: https://huggingface.co/docs/diffusers/index

In two days, we want to kick-start a doc-sprint where we add as many doc-strings as possible. Here the rough draft: https://docs.google.com/document/d/1TwqMlQrjWXi5a0y-C-Wbzx70R4ZQP8f8mq-ARSqB5fI/edit?usp=sharing

Tomorrow we will share more details on what exactly what could be worked on. Let us know if you'd like to participate in the doc-sprint on Wednesday with a :+1:

patrickvonplaten commented 2 years ago

The general doc structure (which is defined here) should look more or less as follows:

"1. Get started" I think the following subsections for "1. Get started" would be nice: "1.1" Diffusers (as is there already), but needs to be adapted "1.2" Quicktour (as is there already), but needs to be adapted “1.3” Installation

“2. Using Diffusers” We only have vision at the moment so let’s maybe just focus on this:

2.1 Loading pretrained instances (explain from_pretrained for all pipelines, models, schedulers)

2.2 Pipelines for Inference 2.1.1 Unconditional Generation 2.1.2 Conditional Generation 2.1.3 Image2Image 2.1.4 In-painting 2.1.5 Create a custom pipeline

"3. Custom Hardware / Optimizing"

pipelines for inference (fp16, autocast, onnx, mps, OpenVino)

“4. Training Diffusers”

3.1 Unconditional Training 3.2 Text-inversion 3.3 Conditional Training (TODO)

“5. Conceptual guides”

"4.1" Here I'd maybe now make a whole section for "Stable Diffusion" with subsections like ethics, optimization, the Hub, ... "4.2" Philosophy (as is there already), but would need to be adapted "4.3" Contribution (we can more or less copy what we have here: )

“5. API”

This is pretty self-explanatory - all public methods need some nice docstring. For now I’d put all models (UNet2D, UNet2DCondition) in one doc string file and also put all schedulers in one doc string file. The each pipeline should get a “Pipeline card” just like we do for Transformers models -> we could just copy-paste the format from: https://github.com/huggingface/diffusers/tree/main/src/diffusers/pipelines/stable_diffusion (which could be the stable_diffusion.mdx model card) to all other models.

patrickvonplaten commented 2 years ago

For section 5.

We need to adapt both the code files, e.g. add nice doc-strings to each public function and class as well as add some .mdx files for the main API classes.

In general section 5 will look as follows:

API
- Main classes 
  - Models (one file with info and docstrings about `ModelMixin`, `Unet2DModel` ,`Unet2DConditionModel`, `VQModel`, `AutoencoderKLModel` 
  - Schedulers (one file with info and docstring about all scheduler classes - cc @natolambert)
  - DiffusionPipeline (one file - only with `DiffusionPipeline` class)
  - Logging (can copy from transformers: https://huggingface.co/docs/transformers/main_classes/logging)
  - Config (one file - only with `ConfigMixin`) and `register_to_config`
  - Outputs (one file with all `BaseOutput` classes)

 - Pipelines (similar in spirit to Transformers models): https://huggingface.co/docs/transformers/model_doc/albert
  - Stable Diffusion (one file which can be a copy of: https://github.com/huggingface/diffusers/tree/main/src/diffusers/pipelines/stable_diffusion)
  - DDPM (we should create a README similar to https://github.com/huggingface/diffusers/tree/main/src/diffusers/pipelines/stable_diffusion
patrickvonplaten commented 2 years ago

Communication tomorrow will happen over a discord and we'll also hold a call at 9h30am Paris time. Please make sure to join discord to join the call and to participate :-)

To join: Follow this link https://discord.gg/bYA2G9yJtP, then click on the 🎨 emoji to get access to the diffusers group. There you'll find the "core-contributions" channel where we'll post all the important info!

natolambert commented 2 years ago

Working on scheduler updates in #376.

patrickvonplaten commented 2 years ago

For anybody who wants to join - ideally we start with 5. API docs for pipelines. Just pick one of the following pipelines and follow the PR: https://github.com/huggingface/diffusers/pull/377/files

Pick one of the following pipelines to get started :rocket: & please note below which pipeline you would like to work on with an open PR :-)

patrickvonplaten commented 2 years ago

Taking latent diffusion in this PR: https://github.com/huggingface/diffusers/pull/377/files

pcuenca commented 2 years ago

I can take stable diffusion if that's ok.

kashif commented 2 years ago

I can take 2 ddpm and pndm

satpalsr commented 2 years ago

I'd like to take latent_diffusion_uncond.

patrickvonplaten commented 2 years ago

Template is ready: https://github.com/huggingface/diffusers/pull/385

=> We can start filling it out :-) (Note that I copy-pasted quicktour.mdx to all doc pages and models.mdx to all API pages to be faster. Make sure to delete / overwrite those :-)

patrickvonplaten commented 2 years ago

First example you can copy from: https://github.com/huggingface/diffusers/pull/377#pullrequestreview-1098963887

kashif commented 2 years ago

i can do ddim karras and score_sde_ve next

natolambert commented 1 year ago

Okay, seems like there is some interest from North America timezone folks! Here's a list of pages that could use some contributions by category. These are a little more open-ended than the Pipelines which focus on one system.

You can see the latest docs here where one can easily find blank pages.

Additionally, doc strings are very important! You can choose some code that no one mentioned above and write docstrings for every public method (I have a lot unmerged yet for schedulers).

Important docstrings:

Doc updates by section

Using diffusers

Loading

Pipelines for inference

These are conceptual guides covering different applications -- could be based off previous colabs released with diffusers:

Special hardware

These may be a bit harder unless you have experience with them (All except fp16 are available, fp16 covered in #404 )

natolambert commented 1 year ago

_toctree had a typo that made the API heading undefined.

Fix here #406 .

patrickvonplaten commented 1 year ago

Thanks!

natolambert commented 1 year ago

Large schedulers push done here in #376, ready for review.

natolambert commented 1 year ago

Did a quick scan for typos in #408

satpalsr commented 1 year ago

@patrickvonplaten Can you check if #413 and #414 look good?

utsavnandi commented 1 year ago

Hi, there is a minor mistake in https://github.com/huggingface/diffusers/blob/main/docs/source/installation.mdx #402

git clone https://github.com/huggingface/diffusers.git cd transformers pip install -e .

The user should cd to diffusers not transformers ! Don't know where to report this, so mentioning this here.

daspartho commented 1 year ago

@utsavnandi Thank you for pointing it out; I've created a PR to address it :) #453