post-registration alerts for packages with similar names (typosquatting)

[ncoghlan]

The question of typosquatting as a threat vector occasionally comes up in discussion of open publication platforms like PyPI, and while I generally agree with this post from npm's Laurie Voss that it's unlikely to be a particularly good attack vector in practice, I still keep an eye out for mitigation ideas that might reduce the threat from "relatively ineffective" to "almost entirely ineffective".

Prompted by this comment https://twitter.com/teabass/status/892708535494217728 about checking the Levenshtein distance between project names, a possibility occurred to me that might actually be viable at scale: rather than notifying the PyPI maintainers, it may be possible to instead notify the maintainers of the packages that have similar names.

While an active alert could make sense for new package name registrations, it could also just be a widget on the project's PyPI page that lists "packages with similar names". For coincidental naming choices, such may help folks discover projects that are genuinely related in some way to the one they're looking at, while for malicious packages, it would significantly reduce the odds of them going unnoticed (especially if the widget noted "first published" dates).

Anyway, I don't think this is urgent (for the reasons stated in Voss's post linked above), but also figured it made sense to share the idea.

[ncoghlan]

Recent incident report from NPM covering a case where downloads of a genuinely malicious package were being induced via typo-squatting: http://blog.npmjs.org/post/163723642530/crossenv-malware-on-the-npm-registry?utm_content=buffer42b52&utm_medium=social&utm_source=twitter.com&utm_campaign=buffer

[ncoghlan]

Report from Slovakia's National Security Authority regarding specific instances of typosquatting specifically on PyPI: http://www.nbu.gov.sk/skcsirt-sa-20170909-pypi/

The folks at https://pytosquatting.overtag.dk also claim to have contacted the PyPI admins about their work, but I don't recall seeing anything from them on any of the relevant mailing lists (i.e. distutils-sig, PSF Infra, PSF Infra Staff).

Maybe they contacted @dstufft or @ewdurbin privately?

[ewdurbin]

i believe there was a report to the PSRT (Python Security Response Team) @ncoghlan, it was only conferred to myself as an instance of packages to be taken down.

[ncoghlan]

Sorry, I conflated two different things in my comment. Victor posted to python-dev regarding the information sharing from the Slovakian security agency, which did indeed come in through PSRT: https://mail.python.org/pipermail/python-dev/2017-September/149569.html

The comment I'm not clear on is the "our primer didn't get reactions from the PyPI admins" one on the Pytosquatting page. Maybe they're confused about the fact that the main organisations they need to be bugging about this topic are the commercial vendors redistributing PyPI clients for profit, rather than the volunteers and non-profit organisations making the platform available for community use?

[dstufft]

No, they're talking about an email they sent to myself, Richard, and Martin that I forgot about because it was PGP encrypted and I first received it on my phone, then it had slipped my mind by the time I was back at my desktop.

[ncoghlan]

Ah, fair enough - that's unfortunate, but it's understandable they might be confused about the lack of response in that case.

In relation to the SK-CSIRT report, I posted a separate issue asking whether or not we might want to make "reserved by admins" clearly distinguishable from other cases of names being reserved: https://github.com/pypa/warehouse/issues/2401

[amirouche]

What is required to go forward regarding this subect?

[brainwane]

Warehouse plans to work on this issue later this year:

systems for the automated detection of malicious uploads will lower the time to response and improve the resiliency of PyPI against attacks such as “pytosquatting”.

[brainwane]

Thanks for starting this issue, Nick! (A few of us discussed it today.)

I'm making a distinction here: #4998 is about typosquatting prevention/detection before/during upload, and is distinct from this issue (which is about notifications, alerts, a "packages with similar names" widget, etc.).

[brainwane]

Blocked on #4998.

[rendorHaevyn]

hi folks, has this found any traction with respect to actual implementation?

I just read an article around this, and noticed a specific user (https://pypi.org/user/wbengtson/) has created over 4,000 typo-squatted packages in order to prevent any attacks in this space.

I'd have thought that pypi or pip etc could simply re-map / re-direct such typos to the correct packages upon pip install x command, with a prompt for user confirmation. At least, as a user, that kind of behaviour would be logical to me.