HPC: LLM from scratch - on prem preparation for Generative AI work

[fmichaelobrien]

see - https://github.com/ObrienlabsDev/blog/wiki/CUDA-based-%E2%80%90-High-Performance-Computing-%E2%80%90-LLM-Training-%E2%80%90-Ground-to-GCP-Cloud-Hybrid#use-cases Start with the following site - read it in its entirety https://jaykmody.com/blog/gpt-from-scratch/

or https://github.com/lm-sys/FastChat

[fmichaelobrien]

review https://github.com/vectara/hallucination-leaderboard

• https://f-a.nz/dev/develop-your-own-llm-like-chatgpt-with-tensorflow-and-keras/

[obriensystems]

RTX-A4500 Dual at 40G

Thinking also of RTX-A6000 with 48G (nvlink) - as an alternative to my two RTX-A4500s with 20+20 = 40G under an nvlink

• https://www.reddit.com/r/deeplearning/comments/11vb220/best_gpus_for_pretraining_robertasize_llms_with_a/
• https://timdettmers.com/2023/01/30/which-gpu-for-deep-learning/
• testing first https://lambdalabs.com/service/gpu-cloud

GTX-4090 Dual at 48G

• https://www.reddit.com/r/LocalLLaMA/comments/15zx322/ideal_setup_for_dual_4090/

Splitting load

• https://huggingface.co/docs/transformers/accelerate
• https://huggingface.co/docs/transformers/perf_infer_gpu_many
• https://vast.ai/
• https://runpod.io/
• https://github.com/TheBlokeAI/dockerLLM
• https://github.com/oobabooga/text-generation-webui