search

Hello-SimpleAI / chatgpt-comparison-detection

Human ChatGPT Comparison Corpus (HC3), Detectors, and more! 🔥
https://arxiv.org/abs/2301.07597
1.25k stars 120 forks source link

The ling detector on Huggingface down? and other issues #25

Open JaySLee opened 1 year ago

JaySLee commented 1 year ago

Dear Hello-SimpleAI,

Thank you for your detectors, which have been valuable resources for my university admin duties. I've posted a similar discussion as this one on Huggingface, hoping one of these will evoke a reply.

  1. I've been making use of all three metrics (one from the single text interface and two from the linguistic interface). Is the 'ling' detector on Huggingface completely offline now? Is it anticipated to be running again at some point in the future?
    • I do see that the 'ling' detector is operational on modelscope and will switch to using it there.
  2. I haven't yet pored over the available data/code carefully. Is the material available enough for me to run your detectors locally (on my own computer)?
  3. The 'ling' detector on modelscope reports "Error" on the following text: "Imagine yourself strolling through the vibrant streets of Mexico City, where a lively celebration is in full swing. The atmosphere is charged with excitement as colors, music, and the exuberant spirit of the Mexican people fill the air. Amidst the bustling crowds, your attention is captivated by an array of national symbols proudly on display. The iconic Mexican flag unfurls, showcasing its bold tricolor of green, white, and red, which dances in the breeze. At the heart of this visual spectacle stands the unmistakable emblem of the Mexican eagle, with its fierce countenance and outstretched wings symbolizing a deep-rooted sense of national pride and identity. Now, let's transport ourselves to the enchanting landscapes and historic cities of the Netherlands, a country renowned for its rich cultural heritage."
    • Removing the phrase ", a country renowned for its rich cultural heritage" from the last sentence produces no Error. This also happens with the single-text detector on modelscope.
  4. The versions of the single text detector on Huggingface and Modelscope seem to be different (with the latter giving same numbers as your roberta detector). I, for one, would prefer the single text detector to remain different from your roberta detector (as the variety of detectors helps in my admin tasks).

Thank you!

Best, Jay