Develop Flask API using Llama 2 Model

[aatish-kayyath]

Flask-based Q&A Web API using Llama 2 Model

Implement a Flask-based web API that accepts user questions and generates answers using the Llama 2 model via the LangChain framework. The API consists of the following:

API Endpoints

• GET /: A simple home route that returns a message indicating the service is for a question-answer generator.

• GET/POST /ask_question:

  • Accepts a question parameter via query string or form data.
  • Processes the question using the Llama 2 model.
  • Returns a generated answer in JSON format.

Technical Details

• Uses CTransformers from the langchain_community module to interact with the Llama 2 model in GGUF format.
• The model is set up to generate answers with configurable parameters like max_new_tokens and temperature.

Future Improvements

• Implement more advanced error handling for invalid or missing questions.
• Add support for multiple LLM models and make the API more flexible.

[aatish-kayyath]

Developed the code for a basic question answering system

[aatish-kayyath]

Developed a Flask API for the same and checked output using Postman

[aatish-kayyath]

Need to update your token using the huggingface-cli login command