[Question] loading Llama3-8B-instruct to HookedTransformer got a warning saying You are not using LayerNorm, so the writing weights can't be centered! Skipping! #653
My gpu can't download checkpoints directly from huggingface.co, I write a function to load models from local directory, then passing to HookedTransformer.
As a result, it jumps a warning, I am a little confused, I raise a few questions:
-whether this function configurations are proper;
-whether this warning influences the results of mechanistic interpretability
The warning will have no impact on your results. Feel free to pass center_writing_weights=False to the from_pretrained call to stop the warning yapping at you
Question
My gpu can't download checkpoints directly from huggingface.co, I write a function to load models from local directory, then passing to HookedTransformer.
As a result, it jumps a warning, I am a little confused, I raise a few questions:
-whether this function configurations are proper;
-whether this warning influences the results of mechanistic interpretability