-
I've been using unstructured for a while in a 100% cpu machine. I've noticed a lot of nvidia files (+2gb) in my venv folder coming from PyTorch (possible one of unstructured's dependencies).
Can I in…
-
I have been finetuning a model based on `Meta-Llama-3-8B` using `finetune`. The model was downloaded from the `meta-llama` Hugging Face. I am running macOS on Apple Silicon. I recently updated llama.c…
-
### What happened?
I try to finetune a llama-like model use `./llama-finetune`.
1. The program works **fine** when I use CPU only.
2. The program causes **segmentation fault** when I use GPU offl…
-
On the firefly board:
The default operating mode of the CPU is interactive, with a frequency of 408000. The default operating mode of NPU is rknpu_ondemand, with a frequency of 1000000000. The defaul…
-
Trying to do inference on arc GPU machine, have followed this guidelines:
```
https://github.com/intel-analytics/ipex-llm/tree/main/python/llm/example/GPU/Pipeline-Parallel-Inference
and run_mi…
-
### System Info
- CPU architecture: x86_64
- CPU memory size: 128G
- GPU name: NVIDIA GeForce GTX 1660S
- GPU memory size: 6G
- TensorRT-LLM branch: main
- TensorRT-LLM commit: 9691e12
- Contai…
-
I was trying to migrate from MLC-LLM to onnxruntime to run Phi-3 on an Orange Pi 5 but I realize that among ALL your execution providers there isn't a single one that takes advantage of the GPU or NPU…
-
### System Info
gpu:
```nvidia-smi
Mon Apr 22 17:00:40 2024
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.161.08 …
-
### System Info
- CPU architecture: x86_64
- GPU properties
- GPU name: NVIDIA A100
- GPU memory size: 40G
- Libraries
- TensorRT-LLM branch or tag: main
- TensorRT-LLM commit: 5d8ca2…
-
### What is the issue?
After going to 0.1.45 from 0.1.43 version I get out of memory, I did try as well
Set-ItemProperty -Path 'HKCU:\Environment' -Name 'OLLAMA_SCHED_SPREAD' -Value 1
and
Set-It…