search

facebookresearch / llm-transparency-tool

LLM Transparency Tool (LLM-TT), an open-source interactive toolkit for analyzing internal workings of Transformer-based language models. *Check out demo at* https://huggingface.co/spaces/facebook/llm-transparency-tool-demo
Other
751 stars 53 forks source link

Load custom model #24

Open SiddhantOjha17 opened 5 months ago

SiddhantOjha17 commented 5 months ago

I have a fine tuned model which is stored locally. As mentioned I will have to make sure it works with TransformerLens. I can confirm I have made it to run perfectly with TransformerLens, now the issue is when I am adding the model to the config file and then running the docker the streamlit UI doesn't open and it just keeps on loading.

A little help would be appreciated on this!!!

SiddhantOjha17 commented 5 months ago

This is another error I came across

KeyError: 'blocks.0.hook_resid_mid' Traceback: File "/home/user/mambaforge3/envs/llmtt/lib/python3.12/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 600, in _run_script exec(code, module.dict) File "/home/user/llm-transparency-tool/llm_transparency_tool/server/app.py", line 699, in app.run() File "/home/user/llm-transparency-tool/llm_transparency_tool/server/app.py", line 683, in run self.run_inference() File "/home/user/llm-transparency-tool/llm_transparency_tool/server/app.py", line 591, in run_inference self._graph = get_contribution_graph( ^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/mambaforge3/envs/llmtt/lib/python3.12/site-packages/streamlit/runtime/caching/cache_utils.py", line 165, in wrapper return cached_func(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/mambaforge3/envs/llmtt/lib/python3.12/site-packages/streamlit/runtime/caching/cache_utils.py", line 194, in call return self._get_or_create_cached_value(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/mambaforge3/envs/llmtt/lib/python3.12/site-packages/streamlit/runtime/caching/cache_utils.py", line 221, in _get_or_create_cached_value return self._handle_cache_miss(cache, value_key, func_args, func_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/mambaforge3/envs/llmtt/lib/python3.12/site-packages/streamlit/runtime/caching/cache_utils.py", line 277, in _handle_cache_miss computed_value = self._info.func(*func_args, *func_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/llm-transparency-tool/llm_transparency_tool/server/utils.py", line 118, in get_contribution_graph return llm_transparency_tool.routes.graph.build_full_graph( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/mambaforge3/envs/llmtt/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/home/user/llm-transparency-tool/llm_transparency_tool/routes/graph.py", line 101, in build_full_graph resid_mid=model.residual_after_attn(layer)[batch_i].unsqueeze(0), ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/mambaforge3/envs/llmtt/lib/python3.12/site-packages/typeguard/init.py", line 1033, in wrapper retval = func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/home/user/llm-transparency-tool/llm_transparency_tool/models/tlens_model.py", line 205, in residual_after_attn return self._get_block(layer, "hook_resid_mid") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/llm-transparency-tool/llm_transparency_tool/models/tlens_model.py", line 189, in _get_block return self._last_run.cache[f"blocks.{layer}.{block_name}"]


File "/home/user/mambaforge3/envs/llmtt/lib/python3.12/site-packages/transformer_lens/ActivationCache.py", line 168, in __getitem__
    return self.cache_dict[utils.get_act_name(key)]
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^