search

Sotera / webpageclassifier

Categorizes a website given URL into one of blog|wiki|news|forum|classified|shopping|undecided.
Apache License 2.0
8 stars 3 forks source link

Improve results #3

Open ctwardy opened 7 years ago

ctwardy commented 7 years ago

Tested on a mix of regular domains and example MEMEX escort/human trafficking domains. Accuracy not great. News and wiki do well. 

  Category                                                URL       Test

0 news nytimes.com news 1 news foxnews.com news 2 news cnn.com news 3 news news.google.com news 4 news npr.org/sections/health-shots/2015/07/16/42326... news 5 news news.google.com/news/story?cf=all&hl=en&ned=us... news 6 news https://www.nytimes.com/2017/01/31/us/politics... news 7 forum ubuntuforums.org forum 8 forum arstechnica.com news 9 forum reddit.com undecided 10 forum ubuntuforums.org/showthread.php?t=937963 forum 11 wiki wiki.ubuntu.com/ForumCouncilAgenda wiki 12 wiki en.wikipedia.org wiki 13 wiki fr.wikipedia.org wiki 14 wiki ru.wikipedia.org wiki 15 wiki ar.wikipedia.org wiki 16 wiki en.wikipedia.org/wiki/denial-of-service_attack wiki 17 blog elegantthemes.com/blog/tips-tricks/how-to-crea... blog 18 blog grahamcluley.com news 19 blog erratasec.com undecided 20 blog krebsonsecurity.com undecided 21 blog joelonsoftware.com undecided 22 blog schneier.com undecided 23 blog troyhunt.com undecided 24 classified craigslist.com undecided 25 classified geebo.com undecided 26 classified backpage.com undecided 27 classified oodle.com undecided 28 classified classifiedads.com undecided 29 classified classifiedsgiant.com undecided 30 classified washingtonpost.com/classifieds undecided 31 classified insidenova.com/classifieds/ undecided 32 classified close5.com/l/arlington-virginia undecided 33 classified meetaphrodite.com undecided 34 classified adultsearch.com/account/signin?origin=%2Fkansa... undecided 35 classified adultsearch.com/kansas/ undecided 36 classified bronx.backpage.com/ undecided 37 classified brooklyn.backpage.com/BodyRubs/sexy-dolls-just... undecided 38 shopping etsy.com/treasury/tags/original+gift undecided 39 shopping overstock.com undecided 40 shopping amazon.com undecided 41 shopping marieclaire.com news 42 shopping ca.boohoo.com undecided 43 shopping walmart.com news 44 shopping grocery.walmart.com undecided 45 undecided ibm.com undecided 46 undecided apple.com undecided 47 undecided nasa.gov undecided 48 undecided jpl.nasa.gov undecided 49 undecided memex.jpl.nasa.gov undecided 50 undecided soteradefense.com undecided 51 undecided google.com undecided

ctwardy commented 7 years ago

Now scoring 0.51.

*ACCURACY*: 29/57 = 0.51

ctwardy commented 7 years ago

So... when to close the ticket?