…dels and models that do not supply the hidden size in the config
Added in a override value for the hidden_size config property that is generally available on GPTQ models.
Created a _get_hidden_size helper function that first checks if the override is set, else looks for the config, and finally if all else fails, generates a set of embeddings and checks the length of the output to get the correct size.
…dels and models that do not supply the hidden size in the config
Added in a override value for the hidden_size config property that is generally available on GPTQ models. Created a _get_hidden_size helper function that first checks if the override is set, else looks for the config, and finally if all else fails, generates a set of embeddings and checks the length of the output to get the correct size.