When I run the first command, everything is fine. But for 2, I encountered the mismatch in weight shape problem:
ValueError: Trying to set a tensor of shape torch.Size([1024, 7168]) in "weight" (which has shape torch.Size([7168, 7168])), this look incorrect.
Does the inference_wizardcoder.py file really support the 33B model?
python inference_wizardcoder.py \ --base_model "WizardLM/WizardCoder-15B-V1.0" \ --input_data_path "./data.jsonl" \ --output_data_path "./result.jsonl"
python inference_wizardcoder.py \ --base_model "WizardLM/WizardCoder-33B-V1.1" \ --input_data_path "./data.jsonl" \ --output_data_path "./result_1.jsonl"
When I run the first command, everything is fine. But for 2, I encountered the mismatch in weight shape problem: ValueError: Trying to set a tensor of shape torch.Size([1024, 7168]) in "weight" (which has shape torch.Size([7168, 7168])), this look incorrect. Does the inference_wizardcoder.py file really support the 33B model?