Limit pocl to <6 (test revert numpy type promotion)

illinois-ceesd / mirgecom

MIRGE-Com is the workhorse simulation application for the Center for Exascale-Enabled Scramjet Design at the University of Illinois.

Other

12 stars 19 forks source link

Limit pocl to <6 (test revert numpy type promotion) #1055

Closed matthiasdiener closed 2 months ago

matthiasdiener commented 3 months ago

Questions for the review:

[ ] Is the scope and purpose of the PR clear?
- [ ] The PR should have a description.
- [ ] The PR should have a guide if needed (e.g., an ordering).
[ ] Is every top-level method and class documented? Are things that should be documented actually so?
[ ] Is the interface understandable? (I.e. can someone figure out what stuff does?) Is it well-defined?
[ ] Does the implementation do what the docstring claims?
[ ] Is everything that is implemented covered by tests?
[ ] Do you see any immediate risks or performance disadvantages with the design? Example: what do interface normals attach to?

inducer commented 3 months ago

What's wrong with pocl 6? This is something I've seen, but it should not affect mirgecom.

matthiasdiener commented 3 months ago

What's wrong with pocl 6? This is something I've seen, but it should not affect mirgecom.

We have seen substantial slowdowns in our prediction cases with both the new type promotion code as well as pocl-5 vs pocl-6: (1 rank, wall time per step [s]):

smoke_test_ks_3d	Porter/GPU	Lassen/GPU	M1/CPU	CI-Linux/CPU
pocl5, old numpy type promo code	0.17	0.18	0.6	1.05
pocl6, old numpy type promo code	0.58	0.47	0.65	1.41
pocl5, new numpy type promo code	0.21	0.19	6.5
pocl6, new numpy type promo code	0.58	0.5	6.5	16.1

For some reason, pocl-6 seems to affect mostly the CUDA devices, while the type promotion code mostly affects the CPU runs.

inducer commented 3 months ago

Yikes. Could you hunt for the simplest example that exhibits the issue and compare the PTX for both? This tool may be able to help. Also, could you file an issue for this? (Since I'm not sure this PR is the best place for the discussion.)