search

argilla-io / distilabel

Distilabel is a framework for synthetic data and AI feedback for engineers who need fast, reliable and scalable pipelines based on verified research papers.
https://distilabel.argilla.io
Apache License 2.0
1.45k stars 111 forks source link

Add serving LLM section in the docs #742

Closed gabrielmbmb closed 3 months ago

gabrielmbmb commented 3 months ago

Description

This PR adds a new section in the docs describing how to leverage a solution like TGI or vLLM to serve an LLM that can be reused across several tasks in the same pipeline using an AsyncLLM client.

codspeed-hq[bot] commented 3 months ago

CodSpeed Performance Report

Merging #742 will not alter performance

Comparing examples-llm-server (af56199) with develop (356a4a3)

Summary

✅ 1 untouched benchmarks