search

flamewei123 / APNEAP-

Codes for paper "Mitigating Privacy Seesaw in Large Language Models: \\ Augmented Privacy Neuron Editing via Activation Patching"
1 stars 0 forks source link

Few clarifications about notations in the paper #1

Open krishnakanthnakkav2 opened 3 months ago

krishnakanthnakkav2 commented 3 months ago

Hi Team,

Can you please clarify the dimensions of (V) and (H_{i})? Assuming the dimension of the LLM is (d) and the number of privacy neurons is (m), how is the dimension of (V) shown as (m \times d)? It is not clear why (d) is involved in the notations. I may be misunderstanding the meaning of "neuron" here.

For example, at the end of each transformer layer, the token is represented with an embedding of dimension (d), and we choose (m) privacy-sensitive positions in this embedding. So, shouldn't (V) or $H$ be a vector of dimension (m)?

Thanks

sev777 commented 1 month ago

Hello, does the code will be released soon?