Performance regression benchmarks

ECP-copa / Cabana

Performance-portable library for particle-based simulations

Other

197 stars 51 forks source link

Performance regression benchmarks #4

Closed sslattery closed 4 years ago

sslattery commented 5 years ago

Capturing outcome of discussion:

[x] A peak flops benchmark (ala GY)
[x] BW bound kernel test
[x] A demonstration of the performance implications of kernel fusion/fission
[x] A demonstration of the performance implications of using multiple lists to track particle properties
[x] An MPM like kernel
[x] A VPIC like PIC kernel
[x] Communication halo exchange/redistribution
[x] Neighbor list generation

guangyechen commented 5 years ago

@sslattery @rfbird Implied I guess also includes testing the basic AOSOA/SOA/AOS with different kernels.

sslattery commented 5 years ago

@guangyechen @rfbird can you link this issue tor relevant PRs you are working on?

sslattery commented 4 years ago

@rfbird Lets clean this one up.

We have both a VPIC-like proxy (CabanaPIC) and an MPM proxy (ExaMPM) in the project space. @streeve has infrastructure to look at performance as a function of multiple particle property lists. The ExaMPM work has a recent publication as well that shows kernel fusion improving results.

Only thing not covered in that list is a bandwidth-bound kernel. Do we have a demo for that? Also can we count these other cases as sufficient to tick the boxes here?

streeve commented 4 years ago

To this end, I recently added a wiki page pointing for the proxy apps - maybe we could add a bit more there about features of note like these.

Is it enough to align the wiki benchmarks and the performance tests?

rfbird commented 4 years ago

I agree this needs to be tidied up/put to bed. Which bits do we still consider a priority?

sslattery commented 4 years ago

To this end, I recently added a wiki page pointing for the proxy apps - maybe we could add a bit more there about features of note like these.

Is it enough to align the wiki benchmarks and the performance tests?

Nice! Yeah I think we need to just align those in the wiki and we are fine.

@rfbird I think the only thing we are missing is bandwidth. I think we have that stream test but does that make sense in the context of what we are trying to achieve?

sslattery commented 4 years ago

Actually looks like bandwidth is covered here: https://github.com/ECP-copa/Cabana/wiki/Benchmarks

So I'll say we are good on this unless there are any objections

rfbird commented 4 years ago

Yea, I think we have everything we need for now. I'm very happy to add anything we need, but we can deal with that as it comes up?

I might look at adding a gather benchmark as that's where I think our performance falls down

sslattery commented 4 years ago

sounds good - lets add benchmarks as we make them