[idea] use the Microsoft Antimalware Scan Interface

[e963db72-c8db-4b8b-b0b5-71a12b2ef945]

BPO	26137
Nosy	@theller, @pfmoore, @tjguk, @zware, @zooba, @ariccio
Dependencies	bpo-27417: Call CoInitializeEx on startup

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = None closed_at = None created_at = labels = ['interpreter-core', 'type-feature', 'OS-windows'] title = '[idea] use the Microsoft Antimalware Scan Interface' updated_at = user = 'https://github.com/ariccio' ``` bugs.python.org fields: ```python activity = actor = 'brett.cannon' assignee = 'none' closed = False closed_date = None closer = None components = ['Interpreter Core', 'Windows'] creation = creator = 'Alexander Riccio' dependencies = ['27417'] files = [] hgrepos = [] issue_num = 26137 keywords = [] message_count = 19.0 messages = ['258455', '258456', '258457', '258458', '258469', '269538', '269570', '269582', '269583', '269585', '269592', '269598', '269600', '269606', '269607', '269622', '269712', '269713', '275397'] nosy_count = 6.0 nosy_names = ['theller', 'paul.moore', 'tim.golden', 'zach.ware', 'steve.dower', 'Alexander Riccio'] pr_nums = [] priority = 'normal' resolution = None stage = None status = 'open' superseder = None type = 'enhancement' url = 'https://bugs.python.org/issue26137' versions = ['Python 3.6'] ```

[e963db72-c8db-4b8b-b0b5-71a12b2ef945]

I'm really not sure what it'd look like, or how it'd work, but CPython should take advantage of Microsoft's Antimalware Scan Interface, which is new to Windows 10. It's designed for applications like interpreters, which can execute u trusted code that may not be visible to antimalware.

[e963db72-c8db-4b8b-b0b5-71a12b2ef945]

See "Windows 10 to offer application developers new malware defenses" https://blogs.technet.microsoft.com/mmpc/2015/06/09/windows-10-to-offer-application-developers-new-malware-defenses/ for an example of how AMSI works with PowerShell.

I think the applicability to CPython is self-evident.

[e963db72-c8db-4b8b-b0b5-71a12b2ef945]

See also: "Security Focus: Defending PowerShell with the Anti-Malware Scan Interface (AMSI)" http://blogs.technet.com/b/poshchap/archive/2015/10/16/security-focus-defending-powershell-with-windows-defender.aspx

[e963db72-c8db-4b8b-b0b5-71a12b2ef945]

When I say "I'm really not sure what it'd look like, or how it'd work" I mean at the C level. At a higher level, there are many places that I imagine are good places to use AMSI: Perhaps expressions passed in from the command line (-c) should be scanned; maybe pip packages should be scanned on installation, and maybe untrusted packages should be scanned right before execution...

Then it's a matter of calling the AMSI APIs in the right places; I don't know where those are.

[zooba]

I actually have a prototype of this already, though I haven't benchmarked the impact yet.

Brett and I were concerned that the applicability wasn't apparent enough given the cost involved, so we've been (slowly) preparing a PEP, mainly to have that record of why it's been added. I'll assign to myself, and keep an eye on python-dev for follow-up.

[zooba]

This now depends on bpo-27417, since we can't enable AMSI without enabling COM, and doing that has a number of back-compat implications.

[pfmoore]

Strong -1 on anything that scans my locally-written scripts by default. There's no reason or justification for that.

Maybe there's a point in having a way to submit an untrusted Python code snippet for scanning, but why would that need to be a core service, as opposed to a 3rd party module?

[zooba]

AMSI is intended for local scanners that are entirely on your own machine, so code never goes anywhere, and everything that passes through the file system is already scanned because of hooks whether you wrote it or not (maybe you're thinking of SmartScreen?).

What this would add is scanning at the exec point in:

python -c exec(decrypt(open('file.bin', 'rb')))

Currently, malware scanners don't get to see the decrypted code, and I'm assured this is a common pattern for getting malware onto machines (yes, in Python).

That said, I fully expect the official releases to require a registry key to enable it (can't be env or CLI option or an attacker would just leave it out :) ). Wouldn't be on for normal use, but would be available for paranoid people.

[pfmoore]

OK, so a 3rd party module providing a "safe_exec" function would make a good proof of concept, I assume. You could probably do that using comtypes or pywin32.

I'm not going to try to say what is or isn't a security threat, that's not my expertise. But I am puzzled as to why "use safe_exec rather than exec" isn't an option, but "use python with the malware scanning option enabled" is. Maybe it's like the Powershell execution policy model, though.

I still don't want it to scan my trusted scripts, though. More interpreter startup overhead? No thanks.

Anyway, thanks for the clarification. It's early days yet to be debating this level of detail, so I'll leave it there.

[zware]

I am puzzled as to why "use safe_exec rather than exec" isn't an option

Because you're going to have a hard time convincing malware authors to use it.

[pfmoore]

> I am puzzled as to why "use safe_exec rather than exec" isn't an option

Because you're going to have a hard time convincing malware authors to use it.

:-) So the malicious payload is the whole python command, not just file.bin. OK, fair enough. But in that case, why hook into exec? The malware author can execute arbitrary Python so doesn't *need* exec.

As I say, though, I'm not an expert in security threats, so I'm OK with accepting that there's a hole here and the proposal plugs it.

[zooba]

So the malicious payload is the whole python command, not just file.bin

Yeah, sorry that wasn't clear. Many vulnerabilities allow attackers to schedule process launches (e.g. via cron/Task Scheduler/etc.) without actually being able to add any files to the machine - Stuxnet took advantage of this, for example. So if Python is already there, you can schedule "python -c "import urllib, base64; exec(...)"" to download->decode->exec arbitrary code without touching the file system or network with obvious sources.

(Right now, I understand base64 is sufficient encryption, at least until the antimalware companies add signatures for base64-encoded scripts. Even then, the slightest customization of the original code is going to break base64 enough to avoid detection, whereas the signatures are flexible enough to handle variations to source code.)

[zware]

But in that case, why hook into exec? The malware author can execute arbitrary Python so doesn't *need* exec.

As I understand it, the malware is distributed in encrypted form (probably encrypted differently each time it propagates) so as to be given a green-light by anti-malware software, then decrypted and run via exec so that the bad code is never actually on disk, and thus never scanned. Yes, the attacker can run arbitrary Python code, but if he just distributed the code in plain text, it could be detected and blocked. The unpacking code is simple and generic enough that it can't be blocked.

As far as actually enabling AMSI, I'm +0. I don't understand it well enough to be +1, and I share Paul's concerns about startup overhead. I'm also unsure that AMSI actually affords any protection: what's to stop the attacker from distributing their own interpreter that just doesn't use AMSI?

[zooba]

what's to stop the attacker from distributing their own interpreter that just doesn't use AMSI?

AppLocker https://technet.microsoft.com/en-us/library/ee619725.aspx

(In short, restrict which executables can be run on a particular system by path/certificate/etc.)

Also a combination of ACLs and the fact that they may not be able to copy files onto the system directly anyway - see my post just before yours.

[zooba]

And in case it's not clear, I *totally* recognize that AMSI is not for everyone. If you're running in a developer environment, your security is almost certainly so lax that it's irrelevant.

However, once you start getting serious about what's deployed on your production machines, Python gets ruled out very quickly because it lacks features like AMSI. Sysadmins don't want to enable a potential attack tool unless it's value drastically outweighs the risks (and on Windows, this is generally/currently not true), or there is sufficient insight into the behavior and use of the tool.

If we want Python to be seen as a production tool and not just a developer tool (which I do), we need to address these concerns.

[pfmoore]

Thanks for the explanation. Based on what's been said, I'd have no objections to this, on a "you don't pay for what you don't use" basis - i.e., users who don't enable AMSI should not pay any cost for its existence.

I'd be extremely happy if Python was viewed as a high-quality option in the server management space, so if this helps in that goal, then that's good.

[e963db72-c8db-4b8b-b0b5-71a12b2ef945]

It's not just Stuxnet, as at least one other Advanced Persistent Threat uses that tactic. An APT (likely Russian intelligence) recently used encoded PowerShell to break into the Democratic National Committe: https://www.crowdstrike.com/blog/bears-midst-intrusion-democratic-national-committee/

From that article:

This one-line powershell command, stored only in WMI database, establishes an encrypted connection to C2 and downloads additional powershell modules from it, executing them in memory.

(As a fun coincidence, they also used py2exe to distribute other modules, which is kinda like a separate interpreter using safe_exec)

[e963db72-c8db-4b8b-b0b5-71a12b2ef945]

We might want to use some kind of Group Policy setting, for the same reason that many Windows security configuration options are there, and that DoD STIGs for Windows https://www.stigviewer.com/stig/windows_8_8.1/ are almost totally about configuring Group Policy settings.

[zooba]

Would be nice to add this, but I have no immediate plans so I'm unassigning it.