-
Hi, I tried to convert a marian model following this [guide](https://opennmt.net/CTranslate2/guides/transformers.html) by using the command
```bash
ct2-transformers-converter --model Helsinki-NLP/op…
-
I'm trying the lit-gpt CPU version on a 32 core server.
I noticed that only one CPU is used.
I tried adding `torch.set_num_threads(num_threads)` but it still uses only 1 CPU.
How can we use multi…
-
PLEASE PLEASE give examples of portions of the instructions where there are none...For example, I want to get statistics using "ctranslate2.ExecutionStats" but I simply can't get it to work in my Pyth…
-
I am unable to run TabbyML with previously downloaded model. Following steps in other reported [issues](https://github.com/TabbyML/tabby/issues/448#issuecomment-1720914534).
The error:
```
[…
-
When I run this:
```python
import transformers
import ctranslate2
model_name = "togethercomputer/RedPajama-INCITE-Chat-3B-v1"
prompt = "Hey what's up how's it going? In this"
tokenizer = …
-
I have downloaded the model from HF and converted the Falcon-40b-instruct using ctranslate2. But when I try to run the model I get two errors
1. the model type is not present in config.json created …
-
Not sure if i am missing something, but when using the new bfloat16 computation types, i get the error:
`expected storage to be of type float16, but is of type bfloat16`
am i missing something or …
-
I'm trying to run a FastAPI web server using `faster-whisper` on 4 different GPUs.
Here is my `TranscribeService` handler that loads 4 times a model on a different GPU using an index to know which …
-
Dependecies all installed according to the documents
```
Traceback (most recent call last):
File "", line 1, in
File "C:\Users\ezrealc\AppData\Local\Programs\Python\Python311\Lib\multiprocess…
-
First, thank you for this awesome work and it indeed improves the transcribe time a lot!
But I'm wondering if it's possible to push to even higher gpu usage so it can be even faster?
From my testing…