[HuggingFace] Make hugging face embeddings work in docker run with multiple pipelines

LangStream / langstream

LangStream. Event-Driven Developer Platform for Building and Running LLM AI Apps. Powered by Kubernetes and Kafka.

https://langstream.ai

Apache License 2.0

393 stars 29 forks source link

[HuggingFace] Make hugging face embeddings work in docker run with multiple pipelines #721

Closed eolivelli closed 1 year ago

eolivelli commented 1 year ago

Summary: when running in docker we use thread to simulate pods and the pipelines share the JVM. There are some issues with PyTorch/DJL and classloading. This patch fixes them and moved the Ollame example to using HuggingFace