Closed egbertwong closed 4 months ago
@egbertwong could you please provide the command you used to reproduce this?
@egbertwong could you please provide the command you used to reproduce this?
Hi, thanks for your reply! I just follow the steps in the README file. And I also write my commands down here:
VARIANT=7b
CKPT_PATH=/mnt/d/Code/gemma/gemma-7b-pytorch/gemma-7b-it-quant.ckpt
sudo usermod -aG docker $USER
newgrp docker
DOCKER_URI=gemma:${USER}
docker build -f docker/Dockerfile ./ -t ${DOCKER_URI}
PROMPT="The meaning of life is"
docker run -t --rm \
--gpus all \
-v ${CKPT_PATH}:/tmp/ckpt \
${DOCKER_URI} \
python scripts/run.py \
--device=cuda \
--ckpt=/tmp/ckpt \
--variant="${VARIANT}" \
--prompt="${PROMPT}"
It seems you are using quantized checkpoint, please make sure that you also add --quant
in the command. I think it should work.
It seems you are using quantized checkpoint, please make sure that you also add
--quant
in the command. I think it should work.
Thanks, it worked!
I use WSL2 on Windows 11 to run gemma_pytorch. The device is i9-13900 + RTX A6000 and I use the 7b-it model. But When I try to run the Gemma interface, I always get an empty result. What might be the reason and how can I solve it?