Open rasbt opened 2 months ago
It seems to be related to the MLP class:
microsoft/phi-2
EleutherAI/pythia-2.8b
stabilityai/stablelm-base-alpha-7b
google/gemma-2-2b
meta-llama/Meta-Llama-3.1-8B-Instruct
openlm-research/open_llama_3b
microsoft/Phi-3-mini-4k-instruct
garage-bAInd/Platypus2-7B
It could be that this could automatically get fixed via #1421
Bug description
For some reason, the tensor parallel implementation generates non-sensical outputs
Expected output (e.g., via base or sequential generation):
What operating system are you using?
Linux
LitGPT Version
Current main branch