Closed hanlint closed 3 months ago
Our modeling_dbrx.py uses nn.LayerNorm(... bias=False) , e.g. https://huggingface.co/databricks/dbrx-instruct/blob/main/modeling_dbrx.py#L642
modeling_dbrx.py
nn.LayerNorm(... bias=False)
This was introduced in Torch 2.1 (e.g. missing from torch 2.0 docs), so bumping the minimum requirements here to torch 2.1.
Fixes #20
hanlint
commented
3 months ago hanlint
commented
3 months ago
Our
modeling_dbrx.pyusesnn.LayerNorm(... bias=False), e.g. https://huggingface.co/databricks/dbrx-instruct/blob/main/modeling_dbrx.py#L642This was introduced in Torch 2.1 (e.g. missing from torch 2.0 docs), so bumping the minimum requirements here to torch 2.1.