-
Hi, I would like to ask you is there any way to use profiler for inference. I was able to run for training, but it didn't work for inference (putting a callback function in predict function).
-
I can see others having success running mistral on a 3090. Am I doing something wrong?
```
Request at 2024-03-29 20:33:44.758325187 -07:00: {"messages":[{"content":"What is the capital of France?"…
-
I am trying to use Llama-2-70b-chat-hf as zero-shot text classifier for my datasets. Here is my setups.
1. vLLM + Llama-2-70b-chat-hf
I used vLLM as my inference engine as run it with:
```
pyt…
-
Right away I want to thank you for this wonderful software that allows to colorize manga on the fly.
I've known about the source app for a while now - https://github.com/qweasdd/manga-colorization-v2…
iG8R updated
2 months ago
-
@dluc , as you are developer of kernel memory, can you provide some sample of MemoryServerless based on LLamaSharp? I am trying to make it work (by getting code for text generator from https://github.…
-
### Trying to run the Shieldgemma model.
The architecture is Gemma2ForCausalLM which should be already supported. The config file specifies the transformers version to be 4.42.4.
I have the foll…
-
### Your current environment
```text
Collecting environment information...
PyTorch version: 2.4.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A
…
-
### Search before asking
- [X] I have searched the YOLOv8 [issues](https://github.com/ultralytics/ultralytics/issues) and [discussions](https://github.com/ultralytics/ultralytics/discussions) and f…
-
**Describe the bug**
> Hi fire! I worked on some of the torchscript stuff so can help out here a bit.
>
> Next step is probably checking that the exported model works as expected. You can verify…
-
Hi.
Testing on AWS g5.2xlarge which has an Nvidia A10G fails with:
```
Serving on http://0.0.0.0:8080.
thread '' panicked at mistralrs-core/src/pipeline/llama.rs:420:17:
Model failed with error `…