Add model invocation layer

[julian-risch]

Problem Statement As an NLP developer and model deployer, I want more choices regarding model management and inferencing (e.g. using remotely or locally deployed models), improving scalability, flexibility, cost-effectiveness, and performance.

User Tasks

• Transparently switch from local runtime model invocation to remote invocation
• Use multiple backends
  • Remote models hosted by other providers (HuggingFace/OpenAI etc)
  • Triton server (probably related: @bogdankostic mentioned that PyTorch 2.0 maps PyTorch models to Triton code)
• Use the same deployed model in multiple different pipelines

[zoltan-fedor]

"Use the same deployed model in multiple different pipelines" >> We achieve this currently with the Ray Serve integration of Haystack.