Closed Qinghao-Hu closed 1 month ago
Thanks for your interesting work!
Could you please provide the configuration of the 8B model?
How do you generate the proxy model? Is it generated based on 8B? Or just a smaller model with arbitrary configuration (e.g., layer num)?
Please see Table 9 here: https://arxiv.org/pdf/2305.10429
The proxy model is just a smaller model (less layers, heads, hidden dims)
Thanks for your interesting work!
Could you please provide the configuration of the 8B model?
How do you generate the proxy model? Is it generated based on 8B? Or just a smaller model with arbitrary configuration (e.g., layer num)?