opendatahub-io / vllm-tgis-adapter

vLLM adapter for a TGIS-compatible gRPC server.
Apache License 2.0
8 stars 11 forks source link

:sparkles: invoke caikit -> peft conversion at load time #161

Closed joerunde closed 1 month ago

joerunde commented 1 month ago

Description

This creates new tempdirs and invokes the caikit to peft conversion utility at prompt adapter load time when a decoder.pt format model is detected.

How Has This Been Tested?

Loaded up onto a dev pod and tested with the bloom_sentiment_1 fixture that was added with the conversion utility. Seems to work fine!

Need to add a unit test for this specifically, still have to get nox up and running locally. Opening this PR now for feedback.

Merge criteria:

codecov-commenter commented 1 month ago

Codecov Report

Attention: Patch coverage is 96.80851% with 3 lines in your changes missing coverage. Please review.

Project coverage is 61.66%. Comparing base (a3620be) to head (95b2064).

Files with missing lines Patch % Lines
...lm_tgis_adapter/tgis_utils/convert_pt_to_prompt.py 0.00% 3 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #161 +/- ## ========================================== + Coverage 58.40% 61.66% +3.26% ========================================== Files 27 28 +1 Lines 1618 1693 +75 Branches 207 208 +1 ========================================== + Hits 945 1044 +99 + Misses 581 553 -28 - Partials 92 96 +4 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.