conda-forge / bitsandbytes-feedstock

A conda-smithy repository for bitsandbytes.
BSD 3-Clause "New" or "Revised" License
1 stars 4 forks source link

Fix CUDA package builds #18

Closed iamthebot closed 1 month ago

iamthebot commented 1 month ago

Summary

Closes #14

tl;dr there was a typo in the build script (s/cuda_compuiler_version/cuda_compiler_version) and also the correct way to specify a CUDA enabled build for this package per the docs is to set -DCOMPUTE_BACKEND=cuda|cpu

Testing

Tested with the code in #14 that was failing (in Python 3.12, CUDA 12 w/ an Nvidia Tesla A10G):

import torch
from transformers import AutoModelForCausalLM, BitsAndBytesConfig

quantization_config = BitsAndBytesConfig(
  load_in_4bit=True,
  bnb_4bit_compute_dtype=torch.float16,
  bnb_4bit_quant_type="nf4",
  bnb_4bit_use_double_quant=True,
)

model_4bit = AutoModelForCausalLM.from_pretrained(
  "facebook/opt-350m", quantization_config=quantization_config
)

Checklist

conda-forge-webservices[bot] commented 1 month ago

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found it was in an excellent condition.

iamthebot commented 1 month ago

@xhochy PTAL, should be all set for review