search

jwkirchenbauer / lm-watermarking

Apache License 2.0
527 stars 66 forks source link

How to calculate ROC curve for watermark detection #46

Open bupt01 opened 1 week ago

bupt01 commented 1 week ago

Hello , I'm having some confusion while calculating the ROC curve for watermark detection and would appreciate your help. Let's say we have 500 watermarked texts to calculate the detection accuracy. Should I calculate the false positive rate by selecting 500 original texts from the dataset, or should I use 500 texts generated by a large model without any watermark?

bupt01 commented 1 week ago

Hello , I'm having some confusion while calculating the ROC curve for watermark detection and would appreciate your help. Let's say we have 500 watermarked texts to calculate the detection accuracy. Should I calculate the false positive rate by selecting 500 original texts from the dataset, or should I use 500 texts generated by a large model without any watermark?