plasma-umass / coz

Coz: Causal Profiling
Other
4.04k stars 160 forks source link

Unable to recreate results #157

Open RN-dev-repo opened 4 years ago

RN-dev-repo commented 4 years ago

We ran COZ on the Swaptions benchmark from Parsec 2.1 with 32 threads. The progress point was set at the same line (HJM_Securities.cpp:99) specified in the published plots . However, our results were different from those published on COZ website (Kindly see the attached figure). Can this be because we are missing something apart from the progress point? The plots were generated by repeating the experiment 100 times. Would also like to know if the COZ_FIXED_LINE option is supported.

Issue_COZ

emeryberger commented 4 years ago

What kind of machine did you run it on? What OS? As the paper states:

We perform all experiments on a 64 core, four socket AMD Opteron machine with 60GB of memory, running Linux 3.14 with no modifications.

Swaptions is notably allocation-intensive (for no good reason, it turns out, but that's an entirely different question), so changes to the malloc implementation could have an impact. In addition, I believe we ran all benchmarks with 64 threads (@ccurtsinger can confirm).

In any event, have you tried merging the patches to see what happens?

RN-dev-repo commented 4 years ago

Experiments were performed on a machine with four AMD Opteron 6282SE eight-core/16-thread CPUs and 128GB of RAM, running Linux 4.15. I will update after merging the patches and running the experiments with 64 threads.

RN-dev-repo commented 4 years ago

Below are the plots for the same experiment with 64 threads. Interestingly, applying the patches provided for Swaptions improved the run time by ~18% and ~15% for 32 and 64 threads respectively. CozProfile_Swaptions_100_64threads