Open Joker-sad opened 6 months ago
It needs to be PyTorch nightly for now.
I use 2.3.0.dev20231214+cu121 version, but I got the error mentioned in https://github.com/pytorch-labs/gpt-fast/issues/49. May it be related with the pytorch version? Which version do you use?
FYI, I am able to run this code repo using torch==2.1.2+cu121
(current stable release, not nightly), by just commenting out torch._inductor.config.fx_graph_cache = True
in generate.py which is not available in this torch version. The rest of code can remain unchanged.
Reference run on RTX 4090
export MODEL_REPO=openlm-research/open_llama_7b
./scripts/prepare.sh $MODEL_REPO
python generate.py --compile --checkpoint_path checkpoints/$MODEL_REPO/model.pth --prompt "Hello, my name is"
Log:
with compile:
This is a good place to start: How to Learn Portuguese (Brazilian) in 3 Simple Steps.
It needs to be PyTorch nightly for now.