-
Firstly, I saved finetuned LORA model as merged_16bit on my huggingface repo. And i have adapter_config.json and adapter_model.safetensor inside my repo. Now when trying to load with AutoAWQForCausalL…
-
I wanted to share the following programs, which I investigated for run times.
These are the verification times:
data/quant_s1.pvl 8.662854194641113 success
data/quant_s2.pvl 7.364089488983154 suc…
-
I got a error when I run the run_awq_llm.sh. I want to ask the question that the llmc can suport the llama3.1-70b? or my config is error. please help me to solve the problem.
my yml file:
base:
…
-
What is best practice to Quantize a common 16Bit 70B Model on a single A100 VM?
What code do you use when you have a 70B e.g. Unsloth fine-tuned model (merged Adapter)?
With the default code I get…
-
Hi
I am getting the following error when aligning my 5' 10X CRISPR scRNAseq data using alevin fry:
/mnt/dv/wid/projects6/Roy-singlecell3/sridharanlab2/data/perturb-seq_on_reprogramming/fastq/G…
-
from trl import SFTTrainer
from transformers import TrainingArguments
from unsloth import is_bfloat16_supported
trainer = SFTTrainer(
model = model,
tokenizer = tokenizer,
train_da…
-
Hi,
I am currently trying to run Taiji on a set of WT and KO RNAseq and ATACseq data. To not mess with previous analyses I decided to use the already existing gene quantification, which I did with …
-
Hi, I am training the LLaMa3 using the SFTtrainer to fit only one data point. The training arguments are shown below:
`base_model = "meta-llama/Meta-Llama-3-8B"
dataset = load_dataset("json", da…
-
config file:
```
base:
seed: &seed 42
model:
type: Mixtral
path: /models/Mixtral-8x7B-Instruct-v0.1
torch_dtype: auto
calib:
name: pileval
download: False
path: …
-
### System Info
### System Info
GPU: NVIDIA A100
Driver Version: 545.23.08
CUDA: 12.3
versions:
https://github.com/NVIDIA/TensorRT-LLM.git (5fa9436) (latest version)
https://github.com/trit…