[P1] Optionally remove the dependency of the config file

[frankaging]

Descriptions: Currently, all the models need to have a config, and the config needs to be inheriting transformer library's config. The model config is only used in,

                intervention = intervention_function(
                    embed_dim=get_dimension(
                        get_internal_model_type(model), model.config, representation
                    ), **other_medata 
                )

to get the components dimension.

We should however allow config-less models where the dimension is directly read-off from the config dict, or dynamically figure out using some helper functions.

• Case 1: Hard code number

  mlp_type_to_dimension_mapping = {
  "block_input": (32,),
  "block_output": (32,),
  "mlp_activation": (32,),
  }

• Case 2: pyvene dynamically figures out the input and output dimensions of all the modules. (checkout torch.compile)

Meanwhile, the intervening component can accept arbitrary model component string e.g. model.h[2].attn.c_proj.output, we can dynamically figure the component out.

[frankaging]

import torch
b_meta = torch.tensor([[1,2,3]], device='meta')
gpt2.to('meta')
print(gpt2(b_meta))

adding notes to track progress. the above meta tensor + a hook can get the intervention module dimension.