Add gpu support for ChatQnA

opea-project / GenAIExamples

Generative AI Examples is a collection of GenAI examples such as ChatQnA, Copilot, which illustrate the pipeline capabilities of the Open Platform for Enterprise AI (OPEA) project.

https://opea.dev

Apache License 2.0

230 stars 148 forks source link

Add gpu support for ChatQnA #308

Closed kding1 closed 3 months ago

kding1 commented 3 months ago

Description

Add gpu support for ChatQnA

Issues

n/a

Type of change

List the type of change like below. Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds new functionality)
[ ] Breaking change (fix or feature that would break existing design and interface)

Dependencies

TGI and TEI images for GPU.

Tests

Verified chatqna on GCP A100 node a2-ultragpu-1g.