search

datarobot / datarobot-user-models

User's custom models boilerplate
Other
96 stars 85 forks source link

[CODEGEN-1924] Add chat hook #1090

Closed baekdahl closed 3 months ago

baekdahl commented 3 months ago

This repository is public. Do not put here any private DataRobot or customer's data: code, datasets, model artifacts, .etc.

Summary

This adds a new hook chat which can be implemented by models to add support for OpenAIs chat completion API. A new route /chat/completions/ route is added to drum server which will invoke this chat hook.

Exception handling in drum server is changed to allow raising HTTPException which will get serialized to JSON. This adds possibility for Custom Model to raise an error when a bad request is received and have that returned as a HTTP 4xx instead of only HTTP 500 response which is currently the case.

Support for Custom Model testing will be added in CODEGEN-1925.

Rationale

We want to add Chat API support for Custom Models so we can improve GenAI applications with support for features like chat history and streaming response.

devexp-slackbot[bot] commented 3 months ago

The Needs Review labels were added based on the following file changes.

Team @datarobot/core-modeling (#core-modeling) was assigned because of changes in files:

custom_model_runner/datarobot_drum/drum/adapters/model_adapters/python_model_adapter.py
custom_model_runner/datarobot_drum/drum/enum.py
custom_model_runner/datarobot_drum/drum/language_predictors/python_predictor/python_predictor.py
custom_model_runner/datarobot_drum/drum/server.py
custom_model_runner/datarobot_drum/resource/components/Python/prediction_server/prediction_server.py
custom_model_runner/datarobot_drum/resource/components/Python/uwsgi_component/uwsgi_serving.py
custom_model_runner/datarobot_drum/resource/predict_mixin.py

Team @datarobot/custom-models (#custom-models) was assigned because of changes in files:

custom_model_runner/datarobot_drum/drum/adapters/model_adapters/python_model_adapter.py
custom_model_runner/datarobot_drum/drum/enum.py
custom_model_runner/datarobot_drum/drum/language_predictors/python_predictor/python_predictor.py
custom_model_runner/datarobot_drum/drum/server.py
custom_model_runner/datarobot_drum/resource/components/Python/prediction_server/prediction_server.py
custom_model_runner/datarobot_drum/resource/components/Python/uwsgi_component/uwsgi_serving.py
custom_model_runner/datarobot_drum/resource/predict_mixin.py
requirements_test_unit.txt
tests/unit/datarobot_drum/drum/adapters/model_adapters/test_python_model_adapter.py
tests/unit/datarobot_drum/drum/test_prediction_server.py

If you think that there are some issues with ownership, please discuss with C&A domain at #core-backend-domain slack channel and create PR to update DRCODEOWNERS\CODEOWNERS file.

akshoop commented 3 months ago

FYI @eric-s-s @scottp-ml for heads up and awareness

kindofluke commented 2 months ago

@baekdahl should https://github.com/datarobot/datarobot-user-models/blob/ac3ca9530d8bd627a2d52964f06a1d07bccc2a96/custom_model_runner/drum_server_api.yaml

this also be updated to have the chat hook?

This is a full openapi specification for the rest interface of DRUM server and is a key reference when implementing clients

baekdahl commented 2 months ago

@kindofluke https://github.com/datarobot/datarobot-user-models/pull/1119

kindofluke commented 2 months ago

Thanks