24/01/24

[junhwi]

https://deepmind.google/discover/blog/alphageometry-an-olympiad-level-ai-system-for-geometry/

https://manifestai.com/blogposts/faster-after-all/

https://www.theverge.com/2024/1/18/24042354/mark-zuckerberg-meta-agi-reorg-interview

Meta’s H100 shipments for 2023 at 150,000, a number that is tied only with Microsoft’s shipments and at least three times larger than everyone else’s. When its Nvidia A100s and other AI chips are accounted for, Meta will have a stockpile of almost 600,000 GPUs by the end of 2024, according to Zuckerberg.

https://huggingface.co/moreh/MoMo-70B-lora-1.8.6-DPO

MoMo-70B is trained using Moreh's MoAI platform, which simplifies the training of large-scale models, and AMD's MI250 GPU.

https://www.reddit.com/r/LocalLLaMA/comments/18xbevs/open_llm_leaderboard_is_disgusting/

https://english.elpais.com/technology/2024-01-19/yann-lecun-chief-ai-scientist-at-meta-human-level-artificial-intelligence-is-going-to-take-a-long-time.html

http://karpathy.github.io/2024/01/21/selfdriving-agi/

https://medium.com/@kunal_79217/an-llm-benchmark-for-financial-document-question-answering-e63e9d2bda25

https://github.com/lucidrains/self-rewarding-lm-pytorch

https://github.com/gabrielchua/RAGxplorer

[hippothewild]

• Self-Rewarding Language Models https://arxiv.org/abs/2401.10020
  • Iterative DPO on LLama 2 70B yields a model that beats Mistral Medium, Claude 2, Gemini Pro, GPT-4 0613 on Alpaca v2 benchmark.
• Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models https://arxiv.org/abs/2401.01335
• WARM: On the Benefits of Weight Averaged Reward Models https://arxiv.org/pdf/2401.12187.pdf
  • Weight Averaged Reward Models (WARM), first finetuning multiple RMs, then averaging them in the weight space.
  • WARM improves efficiency compared to the traditional ensembling of predictions, while improving reliability under distribution shifts and robustness to preference inconsistencies.

[shylee2021]

Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads https://arxiv.org/abs/2401.10774

[seyong92]

Ultra-lightweight Neural Differential DSP Vocoder For High Quality Speech Synthesis https://arxiv.org/abs/2401.10460

DITTO: Diffusion Inference-Time T-Optimization for Music Generation https://arxiv.org/abs/2401.12179