search

eric-mitchell / detect-gpt

DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature
MIT License
349 stars 51 forks source link

About the normalization of perturbation discrepancy #11

Closed cawandmilk closed 1 year ago

cawandmilk commented 1 year ago

Hi. Thank you for your exemplary contribution to the paper and implementation code.

I have a question because the formula mentioned in the paper and the implementation in the code are different.

In line 6 of Algorithm 1 of the paper, there is a part where you divide the square root of the standard deviation $\sigma$ to normalize the perturbation discrepancy $\mathbf{d}$. In the code run.py line 462-462, where this part is implemented, you only calculate the standard deviation. I can't find the amount where we calculate the square root. Could you have intended the z-score standardization to divide by the standard deviation $\sigma$ simply?

Sincerely,

eric-mitchell commented 1 year ago

Thanks for raising this issue- since we're always using the same number of samples, dividing by the standard deviation and dividing by the std err give the same result, modulo a constant scaling factor, so this implementation detail shouldn't affect the results. Feel free to re-open if this doesn't fully address your question!