Open jiahao opened 1 year ago
The current pytorch loader doesn't work on multifile models, which appear to be sharded by rows. For example, in the llama2-13b model directory, w2[1:2560,:] is in consolidated.00.pth and w2[2561:5120,:] is in consolidated.01.pth.
llama2-13b
w2[1:2560,:]
consolidated.00.pth
w2[2561:5120,:]
consolidated.01.pth
The current pytorch loader doesn't work on multifile models, which appear to be sharded by rows. For example, in the
llama2-13bmodel directory,w2[1:2560,:]is inconsolidated.00.pthandw2[2561:5120,:]is inconsolidated.01.pth.