FreedomIntelligence / ChatGPT-Detection-PR-HPPT

Codes and dataset for the paper: Is ChatGPT Involved in Texts? Measure the Polish Ratio to Detect ChatGPT-Generated Text
9 stars 1 forks source link

cool work! Add AI/LLM-gerenated text detection model #3

Closed WangRongsheng closed 9 months ago

WangRongsheng commented 9 months ago

So cool work!

we are also working on something related to AI/LLM-gerenated text detection, can you evaluation our model on this dataset?

  1. English detection model: https://modelscope.cn/models/QiYuan-tech/LLM-Detector-Small-en/summary
  2. Chinese detection model: https://modelscope.cn/models/QiYuan-tech/LLM-Detector-Small-zh/summary
Reasercher-wrs commented 9 months ago
Model Test Set ACC
LLM-Detector-Small-en HPPT/test.json 60.98%
LLM-Detector-Small-en HC3/test.json 83.46%
LLM-Detector-Small-en llama_data/test.json 58.54%
LLM-Detector-Small-zh Chinese_data/polished_abstract/test.json 67.48%

All data is out-of-domain for LLM-Detector! Is HPPT/train.json used to train the model?