Bugfix in custom_monkey_patch for v1 models

johnsmith0031 / alpaca_lora_4bit

MIT License

533 stars 84 forks source link

Bugfix in custom_monkey_patch for v1 models #73

Closed dnouri closed 1 year ago

dnouri commented 1 year ago

Previously generation would fail with:

    File "/alpaca_lora_4bit/text-generation-webui/matmul_utils_4bit.py", line 79, in _matmul4bit_v1_recons
      quant_cuda.vecquant4recons_v1(qweight, buffer, scales, zeros)
  RuntimeError: expected scalar type Half but found Float

See #71