intel / intel-ipsec-mb

Intel(R) Multi-Buffer Crypto for IPSec
BSD 3-Clause "New" or "Revised" License
292 stars 88 forks source link

Fix the clobbering of an XMM register on Windows in sha256_one_block_avx #82

Closed hanblee closed 3 years ago

hanblee commented 3 years ago

Description

The size of _XMM_SAVE was incorrectly set to 7. This PR changes it to 8.

Affected parts

Motivation and Context

On Windows, the stack space for the last (%RSP+0x70) gets clobbered and when the clobbered data gets restored to XMM register, it also gets garbage data.

How Has This Been Tested?

An internal test app which was failing now passes.

Windows using MinGW/gcc

It should not.

Types of changes

Checklist:

pablodelara commented 3 years ago

Code has been merged separately. Thanks!