Descriptions:
Currently, we assume source representations are provided at runtime to do the intervention. This is not easy to swap like zero-out certain representations, adding some noise to certain representations, and swapping with a static yet trainable representation which is fixed for all examples.
We add in a new type of intervention, ConstantSourceIntervention for this line of interventions.
Testing Done:
WARNING:root:Loading trainable intervention from intkey_layer.1.repr.block_output.unit.pos.nunit.1#0.bin.
.Directory './test_output_dir_prefix-d86b4d' created successfully.
.Directory './test_output_dir_prefix-4bf335' created successfully.
WARNING:root:Saving trainable intervention to intkey_layer.0.repr.block_output.unit.pos.nunit.1#0.bin.
WARNING:root:Saving trainable intervention to intkey_layer.1.repr.block_output.unit.pos.nunit.1#0.bin.
.Removing testing dir ./test_output_dir_prefix-19140d
Removing testing dir ./test_output_dir_prefix-d07d02
Removing testing dir ./test_output_dir_prefix-12751f
Removing testing dir ./test_output_dir_prefix-d86b4d
Removing testing dir ./test_output_dir_prefix-4bf335
----------------------------------------------------------------------
Ran 30 tests in 4.461s
OK
Descriptions: Currently, we assume source representations are provided at runtime to do the intervention. This is not easy to swap like zero-out certain representations, adding some noise to certain representations, and swapping with a static yet trainable representation which is fixed for all examples.
We add in a new type of intervention, ConstantSourceIntervention for this line of interventions.
Testing Done: