Closed cdj0311 closed 2 years ago
gpt_example.py
is only used for GPT model. You cannot run GPT-NEOX on it. You can use gptneox_example.cc
to test.
GPT-NEOX does not have position embedding table (model.wpe.bin
).
gpt_example.py
is only used for GPT model. You cannot run GPT-NEOX on it. You can usegptneox_example.cc
to test.GPT-NEOX does not have position embedding table (
model.wpe.bin
).
Do you have any plans to support GPT-NeoX example with Python scripts?
We will consider, but not have such plan now.
hi, I use examples/pytorch/gptneox/utils/eleutherai_gpt_neox_convert.py convert GPT-NeoX model, but there have no model.wpe.bin. However, it must need a model.wpe.bin file when inference with gpt_example.py, such as follows code in gpt.py:
wpe = torch.from_numpy(np.fromfile(ckpt_path + "/model.wpe.bin", dtype=self.weights_data_type)).reshape(-1, self.global_hidden_units) assert self.max_seq_len <= wpe.size(0), ( f"max_seq_len ({self.max_seq_len} must not exceed " f"the value of maximum sequence length during training ({wpe.size(0)})." ) w.append(wpe)