-
I've been serving `codellama/CodeLlama-7b-hf` using openshift AI `Caikit TGIS ServingRuntime for KServe` and trying to interact with it using langchain via `caikit-nlp-client` and [caikit_tgis_langcha…
-
### Description of the feature request:
I'm wondering if it's possible to add support to cancel streaming requests
### What problem are you trying to solve with this feature?
While streaming long…
-
The gapic showcase tests added in [this python PR](https://github.com/googleapis/gapic-generator-python/pull/1764) uncovered an issue with the gapic showcase server. It seems that when using the Strea…
-
### Feature request
Would be nice to have a streaming feature for generation API, so that response would stream token per token and won't wait until full response is generated. gRPC have built-in sup…
Bec-k updated
9 months ago
-
**Is your feature request related to a problem? Please describe.**
I have `google-resumable-media==2.7.1` and want to use it with `httpx`. `httpx` breaks compatibility with `requests` in several w…
-
First of all, thank you for doing a great library development.
It's very cool and easy to use and we would like to incorporate it into our chatbot projects!
But I would like to use the ability to …
-
### Bug Description
When making APi calls langfuse tracing callback in only working when engine / index are initialized again with api call
working in this case:
```python
def stream_gener…
-
**Is your feature request related to a problem? Please describe.**
My API uses server streaming to send message to clients and I can't view the response rate without viewing server logs. This would b…
-
We want to read a request line, execute the query, flush the response and continue to the next request line. This will prevent huge requests to buffer in the application's RAM.
The native go http s…
-
### What happened?
Content Filter Exceptions on Streaming Requests don't get logged to Langfuse
streaming request, Azure raised a content filter error on chunk N
our failure handler did not…