-
### Your current environment
```text
The output of `python collect_env.py`
Collecting environment information...
PyTorch version: 2.3.1+cu121
Is debug build: False
CUDA used to build PyTorch: 12…
-
2023-08-24 11:10:06 | ERROR | stderr | Process model_worker(27196):
2023-08-24 11:10:06 | ERROR | stderr | Traceback (most recent call last):
2023-08-24 11:10:06 | ERROR | stderr | File "D:\env\mi…
-
**Describe the bug**
Creating a docker image to deploy with open ai connection and azure ai search connection.
Note: running this flow from inside VS code works fine when I am logged in (az login, e…
teebu updated
3 weeks ago
-
### System Info
tgi-gaudi 2.0.4
Used below docker compose yaml to launch tgi-gaudi
Serve **llama3.1-70B-instruct model**
--top_k 10
--max_new_tokens 8192
--temperature 0.01
--top_p 0.95
…
-
trying on a MacBook M2
(venv_pdfGPT) ~/pdfGPT/ [main*] lc-serve deploy local api --platform linux/amd64
Traceback (most recent call last):
File "/opt/homebrew/bin/lc-serve", line 5, in
…
-
## Description
I've been using the /generate command to create Jupyter notebooks from text prompts, but it seems to be generating filenames that contain colons (:). This is causing issues, espe…
-
The following run is ok
```
michaelobrien@mbp7 rag % python app.py
USER_AGENT environment variable not set, consider setting it to identify your requests.
docs content size: 43131
LLM P…
-
Repo - [langchain-serve](https://github.com/jina-ai/langchain-serve).
- Exposes APIs from function definitions locally as well as on the cloud.
- Very few lines of code changes and ease of develop…
-
When I send two or more requests to the server, it crashes, error logs below:
CUDA version: 11.7
NVDA Driver Version: 515.65.01
** On entry to SGEMM parameter number 13 had an illegal value
…
-
### Your current environment
accelerate 0.27.2
torch 2.1.2
transformers 4.38.2
pydantic 2.6.1
pydantic_core 2.16.2
pydantic-settings 2.0.3
vllm 0.3.2
xformers 0.0.23.post1
### How you are installing…