-
### Your current environment
```text
Collecting environment information...
PyTorch version: 2.3.1+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A
…
-
config:
```
base:
seed: &seed 42
model:
type: Qwen2
path: /models/Qwen2-7B-Instruct
tokenizer_mode: slow
torch_dtype: auto
calib:
name: pileval
download: False
…
-
I am trying to use apex along with [pytorch/fairseq](https://github.com/pytorch/fairseq/) and using the method for installation as mentioned in the [readme](https://github.com/pytorch/fairseq/#require…
-
Using the same model and loading parameters I'm observing a severe degradation of the model's ability to understand my requests. Could it be related to tensor parallelism? I have to roll back to 0.1.8…
-
## Issue Description ##
I'm trying to attach srsUE to srsENB to [Magma's ](https://github.com/magma/magma) AGW (EPC).
Currently I've managed, what appears to, successfully connect srsenb to the EP…
-
### System Info
### System Info
GPU: NVIDIA A100
Driver Version: 545.23.08
CUDA: 12.3
versions:
https://github.com/NVIDIA/TensorRT-LLM.git (5fa9436) (latest version)
https://github.com/trit…
-
### Context
@arielsvn was looking into a user request in issue https://github.com/AlexsLemonade/refinebio/issues/1996#issuecomment-566190000 .
### Problem or idea
In at least some cases, we h…
-
### Feature request
TGI included marlin kernel is missing padding code
### Motivation
https://github.com/ModelCloud/GPTQModel/issues/328#issuecomment-2408339273
```shell
Now TGI has supported …
-
Directory ./ already exists. Writing into existing directory..
mkdir: cannot create directory ‘.//SCASA_testscasaHNVC02_20230414001259/’: File exists
Preparing for alignment..
Indexing reference.…
-
Hi there,
I cannot load the model RE-DTI-BioGPT by running
```
import torch
from src.transformer_lm_prompt import TransformerLanguageModelPrompt
m = TransformerLanguageModelPrompt.from_pretrain…