google / safeside

Understand and mitigate software-observable side-channels
BSD 3-Clause "New" or "Revised" License
498 stars 54 forks source link

[benchmarks] Add array access microbenchmark #132

Closed zbrid closed 4 years ago

zbrid commented 4 years ago

I'd like to start adding microbenchmarks that folks can run to compare the effects Spectre v1 mitigations have on performance.

To start off with, I wrote a snippet of code that accesses an array 9990 times with in bounds indices and 10 times with out of bounds indices. The goal was to train the branch predictor with the 999 in bounds accesses, so that the out of bounds access would be incorrectly predicted. I'm not 100% sure this is an effective strategy to do that, so let me know if you think there is a better way.

The build creates two versions: a version with no Spectre v1 mitigation and a version with the Speculative Load Hardening (SLH) mitigation enabled. (http://llvm.org/docs/SpeculativeLoadHardening.html)

The numbers I get from the unmitigated version on my machine are around 83,000 for the unmitigated version and 193,000 for the mitigated version which is expected since SLH is known to have a high performance impact.

Microbenchmarks can't give the full picture of performance impact of mitigations since they are out of context by definition. However, I think it's useful to have the data available for discussion. System level benchmarks can come later.

Future plans (anyone can contribute if they'd like):

mmdriley commented 4 years ago

Two CI errors. One is easy -- use kArrLen for a constant.

The other shows me I missed something. We need to take these lines from demos/CMakeLIsts.txt:

# Use C++11 without extensions
set(CMAKE_CXX_STANDARD 11)
set(CMAKE_CXX_EXTENSIONS off)

... and move them to CMakeLists.txt in the root. Put them after project() and before the first add_subdirectory.

Sorry, forgot those were defined in demos/ rather than at toplevel.

googlebot commented 4 years ago

All (the pull request submitter and all commit authors) CLAs are signed, but one or more commits were authored or co-authored by someone other than the pull request submitter.

We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that by leaving a comment that contains only @googlebot I consent. in this pull request.

Note to project maintainer: There may be cases where the author cannot leave a comment, or the comment is not properly detected as consent. In those cases, you can manually confirm consent of the commit author(s), and set the cla label to yes (if enabled on your project).

ℹ️ Googlers: Go here for more info.

zbrid commented 4 years ago

Whoops I had a snafu with git. I'll fix it in a minute.

zbrid commented 4 years ago

Fixed.

googlebot commented 4 years ago

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.