Basic functionality for scheduling interventions to happen on positions not present in the prompt (i.e. generated tokens). Ideally should follow the same procedure for GRU.
Changelog:
timestep_selector, a list of length num_intv of boolean callbacks with signature Callable[[int, torch.Tensor], bool] can be passed to generate() calls. Each intervention calls its callback function with the current position to determine whether the intervention should operate on that position or not.
New handling of Nonevalues in unit locations: If Nones are specified at the batch dimension then interventions are not applied to those examples in the batch.
weird logic where _intervention_getter(), _intervention_setter() functions were being called with single interventions even though they were written to handle an array of intervention keys and return a list of handlers, has been removed
Efficiency and readability improvements in gather_neurons() and scatter_neurons()
Description
Basic functionality for scheduling interventions to happen on positions not present in the prompt (i.e. generated tokens). Ideally should follow the same procedure for GRU.
Changelog:
timestep_selector, a list of lengthnum_intvof boolean callbacks with signatureCallable[[int, torch.Tensor], bool]can be passed togenerate()calls. Each intervention calls its callback function with the current position to determine whether the intervention should operate on that position or not.Nonevalues in unit locations: If Nones are specified at the batch dimension then interventions are not applied to those examples in the batch._intervention_getter(),_intervention_setter()functions were being called with single interventions even though they were written to handle an array of intervention keys and return a list of handlers, has been removedgather_neurons()andscatter_neurons()Testing Done
test_nulling_intervention,test_generation_with_source_intervened_prompt,test_dynamic_static_generation_intervention_parity,test_generation_noopstest_with_subspace_negative,test_scatter_neurons_gpt2_attn_with_head_positiveChecklist:
[Your Priority] Your Title