rth
commented
6 years ago Thanks for your work on handling this incident!
Automated Spam classification for all incoming Projects and Releases
Not an actual classification, but in this notebook I tried to quickly extract links from package description and match them against a blacklist of domain names to see if this would produce anything useful. It turns out it mostly produces false positives so far. Actual classification should work better...
Community crowdsourced classification of spam
Beyond spam labeling, if you are able to provide some dataset with the metadata of packages that were removed as spam (a dataset of valid packages is easier to come by), I think some people in the Python community might be interested in building an ML classifier to automate the detection. This could give you a second evaluation with respect to any solution you implement internally at PyPi..
brainwane
di
Based on information received from the team behind npm, the spam attackers involved in our latest flurry are sophisticated and relentless.
Indeed our initial round of cleanup included 78 Spam User accounts each operating on its own IP Address.
We've added some functionality to the Admin side of things to stop these in their tracks to give us time to assess, but should develop more operational processes moving forward.
I propose the following approach:
Automated Spam classification for all incoming Projects and Releases
Feed the interesting parts of the uploaded metadata for classification by a spam classification model. This should NOT be something that occurs synchronously during the upload, but rather its results should be stored for review by administrators.
Admin interface for review and training of Spam classification results
PyPI Administrators should have a location to review uploads classified as spam. This should allow for the administrators to report back to the model if a given upload was a false positive. It should also allow for administrators to quickly delete true spam.
Community crowdsourced classification of spam
Allow Logged In Users to report spam found on PyPI. This gives us a view of false negative classification. These reports should be rate-limited in order to prevent abuse.
Admin interface for review of User Spam reports
PyPI Administrators should have a location to review User reports of Spam. This should allow for the administrators to report back to the model if a given upload was a false negative. It should also allow for administrators to quickly delete true spam.
Additionally, it should allow for administrators to mark reports as invalid. We may want to keep track of a "reputation" for reporters as well. Users with consistently high reputation or consistently low reputation for reports can be weighted.