Develop LLM Service for Local Model Management

[AKissMail]

The LLM Microservices manage local LLMs such as Meta Llama 3 and specialized models like NB-BERT or NB-GPT-J-6B by the National Library of Norway.

Key responsibilities include:

1. Model Hosting: Host and serve LLM models.
2. API Interface: Provide an interface for the API Gateway and other microservices.
3. Security: Ensure secure handling of model data and responses.
4. Monitoring: Track model performance and usage metrics.

Tasks:

1. Set up local hosting for LLM models.
2. Develop API endpoints for model processing.
3. Implement security measures for data handling.
4. Integrate performance and usage monitoring.