opendatahub-io / vllm-tgis-adapter

vLLM adapter for a TGIS-compatible gRPC server.
Apache License 2.0
8 stars 11 forks source link

Fix bug in example. #159

Closed tdoublep closed 1 month ago

tdoublep commented 1 month ago

Description

Just fixing a small bug in the example, the existing code overwrites the "text" field in the dict, rather than creating a second prompt.

How Has This Been Tested?

before change:

Response contents:
{
  "responses": [
    {
      "generatedTokenCount": 3,
      "text": " for a new",
      "inputTokenCount": 3,
      "stopReason": "MAX_TOKENS"
    }
  ]
}

after change:

Response contents:
{
  "responses": [
    {
      "generatedTokenCount": 3,
      "text": " Nitrogen boils",
      "inputTokenCount": 9,
      "stopReason": "MAX_TOKENS"
    },
    {
      "generatedTokenCount": 3,
      "text": " for a new",
      "inputTokenCount": 3,
      "stopReason": "MAX_TOKENS"
    }
  ]
}

Merge criteria:

codecov-commenter commented 1 month ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 58.35%. Comparing base (a56885c) to head (a401981). Report is 1 commits behind head on main.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #159 +/- ## ======================================= Coverage 58.35% 58.35% ======================================= Files 27 27 Lines 1616 1616 Branches 270 270 ======================================= Hits 943 943 Misses 581 581 Partials 92 92 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.