Improvements for beginners

[TheFusion21]

• Add Documentation to set up a training loop
• Support for webdataset
• slurm (as people wanted it in the LAION discord)
• Docs for advanced parameters
• Requirements (for small scale and large scale training)
• preencoding ( ability to use seperate Cuda device )

[lucidrains]

yea, all good points

you want to take on any of this?

[lucidrains]

ok, let's make this a community effort. I'll get around to this after I wrap up all the remaining video todos

[TheFusion21]

Will do preencoding as I was planing on doing it just haven't had time. Docs for training can do. maybe add a wiki? Requirements should be easy software and hardware

Slurm don't know enough Webdataset maybe have to take a look into it

[lucidrains]

@TheFusion21 yea, i'm stretched for time too

i really want to get started on some exciting new protein diffusion work, which will take up a month at least

[lucidrains]

@TheFusion21 no hurries, we can just keep this open and knock out items on the list when time allows

[jacobwjs]

Will do preencoding as I was planing on doing it just haven't had time. Docs for training can do. maybe add a wiki? Requirements should be easy software and hardware

Slurm don't know enough Webdataset maybe have to take a look into it

@TheFusion21 Did you ever get around to work on precomputing text embeddings?