-
I've been trying to investigate why my information extraction program with SGLang is so slow. I've rented RTX3090 (1 x RTX 3090, 6 vCPU 26 GB RAM) and H100 (1 x H100 SXM, 16 vCPU 125 GB RAM) on RunPod…
-
### Reminder
- [X] I have read the README and searched the existing issues.
### Reproduction
pip install tiktoken
pip install transformers_stream_generator einops
deepspeed --num_gpus 2 ../../src…
-
Following the guide, i manage to install all requirements in a brandy new conda env. I tried to run the zebra example (in which i am interested the most) got no segmentation results. I had tried othe…
-
### Checked other resources
- [X] I added a very descriptive title to this issue.
- [X] I searched the LangChain documentation with the integrated search.
- [X] I used the GitHub search to find a…
-
I'm finding this repo to be a user friendly, extensible, memory efficient solution for training/fine-tuning models. However, when it comes to inference, there is a usability gap that could be solved b…
-
Hi, I tried using SaT as a drop in for WtP (wtp-canine-s-1l-no-adapters).
However no matter which variant between 1l and 3l I try, it always takes nearly a second to run inference (vs 0.013s (cpu) …
-
The existing tokenizer implementation supports only GPT models. The [Microsoft.ML.Tokenizers](https://www.nuget.org/packages/Microsoft.ML.Tokenizers/0.21.0-preview.22621.2) package provides a …
-
I created a new environment and ran pip install crewai and pip install 'crewai[tools]'. However, every time I run the code, I encounter the same error. I've tried reinstalling, creating another enviro…
Vigtu updated
3 weeks ago
-
**Describe the bug**
Error when using this:
```python
logging.basicConfig(format="%(levelname)s - %(name)s - %(message)s", level=logging.WARNING)
logging.getLogger("haystack").setLevel(logging.DE…
-
(openvino.genai) E:\projects\openvino.genai\text_generation\causal_lm\cpp>python ..\..\..\llm_bench\python\convert.py --model_id TinyLlama/TinyLlama-1.1B-Chat-v1.0 --output_dir .\TinyLlama-1.1B-Chat-v…