Research spike: Boring Bomb

[adhurjaty]

The Boring Bomb will get the contents from the news sites, generate a new headline, and update the database

• What kind of service is this? Cron job within the backend, separate lambda, CLI?
• What is needed to connect it with an LLM?

[maxtriano]

For serving LLMs I know of vLLM, which allows deploying an LLM of your choosing as a server that implements the OpenAI API https://docs.vllm.ai/en/latest/getting_started/quickstart.html. Has support for most popular models on HuggingFace (and a lot of fancier features that don't really matter for MVP, like speculative decoding)