search

discopop-project / discopop

DiscoPoP - Discovery of Potential Parallelism
http://www.discopop.org
BSD 3-Clause "New" or "Revised" License
33 stars 19 forks source link

Inquiry about using Discopop for Fortran code analysis #542

Closed Masterwater-y closed 2 months ago

Masterwater-y commented 5 months ago

I was wondering if it is possible to use Discopop to analyze Fortran code, as I only see C and CXX wrappers in the discopop/build/scripts directory. However, in the experimental section of the paper "Unveiling parallelization opportunities in sequential programs," I noticed that Discopop was used to analyze NPB3.3.1, which consists mostly of Fortran code. I would like to inquire about how to use Discopop to analyze NPB or Fortran code.

Thank you for your time and assistance.

lukasrothenberger commented 5 months ago

Hello @Masterwater-y,

Since the DiscoPoP profiler is built upon LLVM-IR, analyzing Fortran code is theoretically possible. However, i have to admit that for a while now no one in our team is "fluent" in Fortran, which is why it has been neglected a bit, and i can not guarantee that it will work out of the box, simply because we have not tested it in some time. The same is true for the analysis steps after the profiling.

Nevertheless, i would be more than happy to help! I will try instrumenting a simple benchmark and get back to you asap.

Best Lukas

lukasrothenberger commented 5 months ago

Hello again,

a possible approach to instrument a Fortran application is to create a llvm-ir .ll file from it using the following flags -g -O0 -fno-discard-value-names -S -emit-llvm, and apply the DiscoPoP instrumentation pass located in <build>/libi/LLVMDiscoPoP.so named -DiscoPoP via opt.

Unfortunately, my environment does not allow this optimization out of the box.

Masterwater-y commented 5 months ago

@lukasrothenberger Thank you for your response, I'll try the solution. I would like to ask how your team used Discopop to experiment with NPB, as it contains a lot of Fortran code. Did you use the C version implementation or the solution you provided above?

lukasrothenberger commented 5 months ago

Hello @Masterwater-y, The wrappers are a fairly recent addition to the "ecosystem", which is why they do not support fortran (yet). Nevertheless, it should be possible to instrument Fortran code manually. Unfortunately, i am not aware of any documentation for the mentioned case. My guess is a mix of both, since even C code needed to be manually instrumented up until fairly recently.

Masterwater-y commented 5 months ago

Hello @lukasrothenberger I have made some attempts in the past few days and encountered some problems. Firstly, I installed flang from llvm-project, but the flang from llvm11 failed to generate LLVM IR correctly and reported the following error.

flang -S -g -O0 -emit-llvm square.f90 gfortran: warning: square.o: linker input file unused because linking not done

Then I installed the latest llvm and the corresponding flang, which is called flang-new, and it can generate LLVM IR correctly.

I ran the following command and encountered some errors. I want to know if I misunderstood something or if there is a problem with the command I ran.

flang-new -g -O0 -fno-discard-value-names -S -emit-llvm -o output.ll ff.f90 flang-new: error: unknown argument: '-fno-discard-value-names'

flang-new -g -O0 -S -emit-llvm -o output.ll ff.f90 opt -load ../build/libi/LLVMDiscoPoP.so -DiscoPoP -o output_inst.ll output.ll

PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace. Stack dump:

  1. Program arguments: opt -load ../build/libi/LLVMDiscoPoP.so -DiscoPoP -o output_inst.ll output.ll

    0 0x000055d249b5f79f llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/home/yhl/pkg/llvm17-root/install/bin/opt+0x2f5879f)

    1 0x000055d249b5ccc4 SignalHandler(int) Signals.cpp:0:0

    2 0x00007f9ec4936420 __restore_rt (/lib/x86_64-linux-gnu/libpthread.so.0+0x14420)

    3 0x00007f9ebfd49485 (/usr/lib/llvm-11/lib/libLLVM-11.so.1+0xa89485)

    4 0x00007f9ebfd493db llvm::raw_ostream::write(char const*, unsigned long) (/usr/lib/llvm-11/lib/libLLVM-11.so.1+0xa893db)

    5 0x00007f9ebfca8f7f (/usr/lib/llvm-11/lib/libLLVM-11.so.1+0x9e8f7f)

    6 0x00007f9ebfca912e (/usr/lib/llvm-11/lib/libLLVM-11.so.1+0x9e912e)

    7 0x00007f9ebfc9d899 (/usr/lib/llvm-11/lib/libLLVM-11.so.1+0x9dd899)

    8 0x00007f9ebfb8f4e1 (/usr/lib/llvm-11/lib/libLLVM-11.so.1+0x8cf4e1)

    9 0x00007f9ec496fb9a (/lib64/ld-linux-x86-64.so.2+0x11b9a)

    10 0x00007f9ec496fca1 (/lib64/ld-linux-x86-64.so.2+0x11ca1)

    11 0x00007f9ec44f6ba5 _dl_catch_exception (/lib/x86_64-linux-gnu/libc.so.6+0x160ba5)

    12 0x00007f9ec497443d (/lib64/ld-linux-x86-64.so.2+0x1643d)

    13 0x00007f9ec44f6b48 _dl_catch_exception (/lib/x86_64-linux-gnu/libc.so.6+0x160b48)

    14 0x00007f9ec497360a (/lib64/ld-linux-x86-64.so.2+0x1560a)

    15 0x00007f9ec494634c (/lib/x86_64-linux-gnu/libdl.so.2+0x134c)

    16 0x00007f9ec44f6b48 _dl_catch_exception (/lib/x86_64-linux-gnu/libc.so.6+0x160b48)

    17 0x00007f9ec44f6c13 _dl_catch_error (/lib/x86_64-linux-gnu/libc.so.6+0x160c13)

    18 0x00007f9ec4946b59 (/lib/x86_64-linux-gnu/libdl.so.2+0x1b59)

    19 0x00007f9ec49463da dlopen (/lib/x86_64-linux-gnu/libdl.so.2+0x13da)

    20 0x000055d249b4719e llvm::sys::DynamicLibrary::getPermanentLibrary(char const, std::__cxx11::basic_string<char, std::char_traits, std::allocator>) (/home/yhl/pkg/llvm17-root/install/bin/opt+0x2f4019e)

    21 0x000055d249ab9381 llvm::PluginLoader::operator=(std::__cxx11::basic_string<char, std::char_traits, std::allocator> const&) (/home/yhl/pkg/llvm17-root/install/bin/opt+0x2eb2381)

    22 0x000055d2471df3af llvm::cl::opt<llvm::PluginLoader, false, llvm::cl::parser<std::__cxx11::basic_string<char, std::char_traits, std::allocator>>>::handleOccurrence(unsigned int, llvm::StringRef, llvm::StringRef) (/home/yhl/pkg/llvm17-root/install/bin/opt+0x5d83af)

    23 0x000055d249a7f86f llvm::cl::ParseCommandLineOptions(int, char const const, llvm::StringRef, llvm::raw_ostream, char const, bool) (/home/yhl/pkg/llvm17-root/install/bin/opt+0x2e7886f)

    24 0x000055d2471e22b6 optMain (/home/yhl/pkg/llvm17-root/install/bin/opt+0x5db2b6)

    25 0x00007f9ec43ba083 __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x24083)

    26 0x000055d2471dab7e _start (/home/yhl/pkg/llvm17-root/install/bin/opt+0x5d3b7e)

    Segmentation fault (core dumped)

lukasrothenberger commented 5 months ago

Hello @Masterwater-y, your approach and the used command in the second try should be correct, as far as i can tell. I've tried it myself, and ran into issues due to version mismatches of llvm-17's flang-new, or rather opt, and the LLVMDiscoPoP.so, since it was compiled using llvm-11.

I suspsect the only true solution to solve the issue is to update the Instrumentation pass to use llvm-17, then it should be compatible with opt-17 and flang-new. Unfortunately, we do not have the time to tackle this at the moment, as this change would require some major changes. I am really sorry for this.

Perhaps it is possible to get it working for a specific combination of clang, flang and manual linking, but i can not guarantee anything.