Forward get_route_info and ROUTE_INFO_HEADER_KEY from backend

caikit / caikit-nlp

Apache License 2.0

12 stars 45 forks source link

Forward get_route_info and ROUTE_INFO_HEADER_KEY from backend #364

Closed gabe-l-hart closed 2 months ago

gabe-l-hart commented 2 months ago

Still TODO:

[x] Bump caikit-tgis-backend once https://github.com/caikit/caikit-tgis-backend/pull/57 is merged

gabe-l-hart commented 2 months ago

NOTE to self: Use MODEL_MANAGER.get_module_backends here

gabe-l-hart commented 2 months ago

I'm retracting my NOTE to self because using MODEL_MANAGER.get_module_backends would potentially result in different semantics if there are multiple TGISBackend instances configured.