Problems while running tests

[righier]

Hello, I find this project to be quite interesting and I've been wanting to perform some tests with the purpose of expanding on this work. But currently I have not been able to run the tests included inside the repository.

I managed to build CPF on Ubuntu 20.04 on the latest master commit (using NOELLE and SCAF, both on the latest master commit), but I'm having issues when trying to run any of the tests included.

I'm not sure which is the correct command I should invoke, but I tried these:

• make compare.out (while inside the directory cpf/tests/regression/2mm/src) This gives the following error:

  Postprocessor step 1
  opt -load ... -spec-priv-postprocess-1 -dce -stats benchmark.collaborative-pipeline-nopeep.bc -o tmp1.bc
  Error opening '.../noelle-install-release/lib/Loops.so': .../noelle-install-release/lib/Loops.so: undefined symbol: _ZN7liberty6LoopAA2IDE
  -load request ignored.
  opt: for the   --stat option: may only occur zero or one times!
  opt: Unknown command line argument '-spec-priv-postprocess-1'.  Try: 'opt --help'
  opt: Did you mean '  --spec-priv-classify'?

It seems like llvm is being called with the parameter '-spec-priv-postprocess-1', which is not recognized. And there are some linking problems with NOELLE. Maybe I'm using the wrong NOELLE version? I previously tried using NOELLE 9.6.0, but I got some different error.

• make benchmark.compare.out (also from inside cpf/tests/regression/2mm/src) Same error

• python ${CPF}/tests/scripts/GetResults.py -p ${CPF}/tests/regression -b ${CPF}/tests/regression/bmark_list_reg.json -r 0 -n 7 -s reg_all -f This command runs with no errors, but I'm not completely sure if it's working as expected. If I look at the log file '2mm.benchmark.collaborative-pipeline.dump' I can still see an error: Error opening '.../noelle-install-release/lib/Noelle.so': .../noelle-install-release/lib/Noelle.so: undefined symbol: _ZN4llvm6noelle15LoopTransformer2IDE But if I look at 'status_2mm.json' it seems like it managed to build and execute the parallel version:

  {
  "Edge": true,
  "Loop": true,
  "LAMP": true,
  "SpecPriv": true,
  "Experiment":
  {
      "speedup": 14.64,
      "worker_cnt": 22,
      "total_coverage": 100.0,
  ...

What I'm trying to do is to at least be able to build the binaries for a test (both sequential and parallel) and then run them and measure the achieved speedup.

Thank you for providing this amazing tool!

[vgene]

Hi righier,

Thank you for your interests in this work! The current master is under some major development. If you want to try out the parallelization on your machine, please try to use the v9.0.0 branch and the corresponding SCAF and NOELLE (both v9.0.0 in this case). The command make benchmark.compare.out should work on that branch and generate corresponding binaries and run on a X64 machine.

The error you are seeing are related to the postprocessing (codegen) steps and actually generating the binaries, which is not a part of the regression test. The printed out "speedup" in the status_2mm.json is the estimated speedup from CPF. For the next couple of months, our development focus will not be on the codegen part so we do expect the master branch to fail on generating binaries. (Instead, we are focusing on the optimization and parallelism discoveries for now) We will come back to fix the codegen part once we finish what's on the plate.

Best, Ziyang

[righier]

Thank you for your swift response, I really appreciate it!

I just tried using the v9.0.0 version of the repository and I managed to build it successfully, but I still incurred in an error while building the individual tests. While running this command:

time opt benchmark.bc -load /home/ermanno/cpf-workspace/cpf-install-release/lib/libUtil.so -load /home/ermanno/cpf-workspace/scaf-install-release/lib/libSCAFUtilities.so -load /home/ermanno/cpf-workspace/noelle-install-release/lib/libSvf.so -load /home/ermanno/cpf-workspace/noelle-install-release/lib/libCudd.so -load /home/ermanno/cpf-workspace/scaf-install-release/lib/libMemoryAnalysisModules.so -load /home/ermanno/cpf-workspace/cpf-install-release/lib/libLoopProf.so -load /home/ermanno/cpf-workspace/cpf-install-release/lib/libMetadata.so -load /home/ermanno/cpf-workspace/cpf-install-release/lib/libLAMPLoad.so -load /home/ermanno/cpf-workspace/cpf-install-release/lib/libLAMP.so -load /home/ermanno/cpf-workspace/cpf-install-release/lib/libRedux.so -load /home/ermanno/cpf-workspace/cpf-install-release/lib/libPointsToProfiler.so -load /home/ermanno/cpf-workspace/cpf-install-release/lib/libGraphAlgorithms.so -load /home/ermanno/cpf-workspace/noelle-install-release/lib/AllocAA.so -load /home/ermanno/cpf-workspace/noelle-install-release/lib/BasicUtilities.so -load /home/ermanno/cpf-workspace/noelle-install-release/lib/DataFlow.so -load /home/ermanno/cpf-workspace/noelle-install-release/lib/LoopStructure.so -load /home/ermanno/cpf-workspace/noelle-install-release/lib/TalkDown.so -load /home/ermanno/cpf-workspace/noelle-install-release/lib/PDGAnalysis.so -load /home/ermanno/cpf-workspace/cpf-install-release/lib/libGraphAlgorithms.so -load /home/ermanno/cpf-workspace/noelle-install-release/lib/Architecture.so -load /home/ermanno/cpf-workspace/noelle-install-release/lib/HotProfiler.so -load /home/ermanno/cpf-workspace/noelle-install-release/lib/Loops.so -load /home/ermanno/cpf-workspace/cpf-install-release/lib/libStrategy.so -load /home/ermanno/cpf-workspace/cpf-install-release/lib/libOrchestration.so -load /home/ermanno/cpf-workspace/cpf-install-release/lib/libSpeculation.so -load /home/ermanno/cpf-workspace/noelle-install-release/lib/Heuristics.so -load /home/ermanno/cpf-workspace/noelle-install-release/lib/ParallelizationTechnique.so -load /home/ermanno/cpf-workspace/noelle-install-release/lib/DOALL.so -load /home/ermanno/cpf-workspace/cpf-install-release/lib/libCodeGen.so -lamp-inst-cnt -lamp-map-loop -lamp-load-profile -basic-loop-aa -scev-loop-aa -auto-restrict-aa -intrinsic-aa -global-malloc-aa -pure-fun-aa -semi-local-fun-aa -phi-maze-aa -no-capture-global-aa -no-capture-src-aa -type-aa -no-escape-fields-aa -acyclic-aa -disjoint-fields-aa -field-malloc-aa -loop-variant-allocation-aa -std-in-out-err-aa -array-of-structures-aa -kill-flow-aa -callsite-depth-combinator-aa -unique-access-paths-aa -llvm-aa-results -basicaa -globals-aa -cfl-steens-aa -tbaa -scev-aa -cfl-anders-aa -objc-arc-aa -scoped-noalias -veto -nander -read-specpriv-profile -remed-selector -spec-priv-preprocess -spec-priv-apply-separation-spec -spec-priv-apply-control-spec -spec-priv-apply-value-pred-spec -specpriv-mtcg -stats -o benchmark.collaborative-pipeline-nopeep.bc

It seems to end up in segmentation fault, here is the full log: log.txt

Maybe this issue is related to #32? Have you or @andreybokhanko ever found a fix to that problem?

Thank you again for your help, and sorry for bothering you

[vgene]

Thanks for identifying this. I'll try to reproduce it and fix it this week.

We are happy that you are using CPF and reproduce the results (maybe even explore other usage with it later!). Please let us know if there's any issue/potentials that you see while building/using it. If it's helpful, we can also setup virtual help sessions later.

[vgene]

I've reproduced the problem on my local environment. Working on a fix now.

[vgene]

Hi,

Please try to apply this patch (some bug fixes later than v9.0.0) by git apply diff.txt under cpf then remake CPF by make cpf under the cpf bootstrap. Then you can try make compare.out and make seq.time and make parallel.time.

All instruction above is still assuming using version v9.0.0. We will try to bring master ready for binary generation soon.

[vgene]

@righier Just a followup, is the problem addressed with the patch?

[righier]

Hi vgene, thank you for the patch, Last week I tried it and indeed now 2mm and 3mm are compiling (I haven't tried all the other tests yet), now I can see the file benchmark.collaborative.exe which (correct me if I'm wrong) I think is supposed to be the parallel executable. The problem I'm facing now is that when I run it with the provided arguments: ./benchmark.collaborative.exe 0 1024 1024 1024 1024 (2mm test) It just keeps running indefinitely (I tried to let it run for a few hours but no luck)

I was planning to rerun the tests in debug mode to try and figure out what is going on but I haven't had the time to do that yet

[vgene]

Please use the debug build of the runtime (by source ${install_root_dir}/cpf-env-debug.rc) and rerun the make parallel.time. This is a known issue but as I mentioned before, we don't yet have time to fix the runtime :(

[righier]

I just tried the debug version and it is working now for some of the tests (2mm, 3mm, covariance, correlation) This should be enough for my purposes. Thank you a lot for your help!