search

ashwinktpu / StarPlat

16 stars 16 forks source link

readme

GraphDSL

CUDA Branch

StarPlat: A Versatile DSL for Graph Analytics

- Please use WorkingBranch

GraphDSL MPI Backend

How to compile Generated MPI Codes on Aqua Cluster

$ module load openmpi316
$ mpicxx -I/lfs/usrhome/oth/rnintern/scratch/MPI_Comparison/boost/install_dir/include program.cc ../boost/install_dir/lib/libboost_mpi.a ../boost/install_dir/lib/libboost_serialization.a -o output  

(Assuming your parent directory contains boost library files)
Done with the compilation!

How to run MPI programs

$ /lfs/sware/openmpi316/bin/mpirun -np 64 -hostfile $PBS_NODEFILE $PBS_O_WORKDIR/output /lfs1/usrscratch/phd/cs16d003/11suiteDSL/weightedGraphs/sinaweibowt.txt > $PBS_O_WORKDIR/output_logfile_name.txt 

(The CS16d003 contains all the graphs which we are using)
Done with running!

Sample Script to run on Aqua Cluster

#!/bin/bash
#PBS -o logfile.log
#PBS -e errorfile_slash.err
#PBS -l walltime=00:60:00
#PBS -l select=2:ncpus=32
#PBS -q rupesh_gpuq

module load openmpi316

#tpdir=`echo $PBS_JOBID | cut -f 1 -d .`
#tempdir=$HOME/scratch/job$tpdir
#mkdir -p $tempdir
#cd $tempdir

#Compilation not needed already done
#mpicxx -I/lfs/usrhome/oth/rnintern/scratch/MPI_Comparison/boost/install_dir/include triangle_count_dsl.cc ../boost/install_dir/lib/libboost_mpi.a ../boost/install_dir/lib/libboost_serialization.a -o tc_exe

#Execution
/lfs/sware/openmpi316/bin/mpirun -np 64 -hostfile $PBS_NODEFILE $PBS_O_WORKDIR/sssp_exe /lfs1/usrscratch/phd/cs16d003/11suiteDSL/weightedGraphs/sinaweibowt.txt > $PBS_O_WORKDIR/output_sinaweibo.txt

#mv ../job$tpdir $PBS_O_WORKDIR/.

Sample main() for generated CC (SSSP)

int main(int argc, char* argv[])
{

  mpi::environment env(argc, argv);
  graph G1(argv[1]);
  int src=0;
  int* dist;
  Compute_SSSP(G1,dist,src);

  return 0;
}

Aqua Cheats

$ qsub script.cmd : To run/subscribe script on specified queue
$ qstat : Gives the status of scripts running 
$ qdel <processid> : Removes script from queue
Branches Build Status Generation
Working Working Branch Working Branch Working Branch //TODO
CUDA CUDA Branch //TODO
OMP OMP Branch //TODO
MPI MPI Branch //TODO

How to build


# Step 1. Clone the repo from our Main branch
git clone -b WorkingBranch https://github.com/nibeditabh/GraphDSL.git

# Step 2. Create a StarPlat compiler
cd GraphDSL/src
make

How to Build (Mac OS | Tested on M1)

By default, MacOS comes with the clang compiler, which can cause syntax errors when running the make command due to differences between gcc and clang semantics. To resolve this, follow these steps:

Installation:

  1. Install gcc using Homebrew:

    brew install gcc

  2. Check the installation location of the g++ binaries:

    which brew
ll /opt/homebrew/bin/g++-*

  3. Create symbolic links to the installed version, (presumes that the above command returns gcc-13):

    sudo ln -sf /opt/homebrew/bin/gcc-13 /usr/local/bin/gcc
sudo ln -sf /opt/homebrew/bin/g++-13 /usr/local/bin/g++
sudo ln -sf /opt/homebrew/bin/c++-13 /usr/local/bin/c++
sudo ln -sf /opt/homebrew/bin/cpp-13 /usr/local/bin/cpp

    Note: Modifying anything in /usr/bin/* is discouraged and may require disabling SIP (System Integrity Protection). Therefore, we create symlinks to /usr/local/bin/*.

  4. After installation, check the versions:

    g++ --version # Should return clang version
g++-13 --version # Should return (Homebrew GCC)

  5. Update the Makefile (StarPlat/src/Makefile) to use the symlinked g++:

    CC = /usr/local/bin/g++

  6. Add %token return_func in the file lrparser.y (StarPlat/src/parser/lrparser.y)

How to generate

# Step 3. Generate files from DSL files.  
./StarPlat [-s|-d] -f <dsl.sp> -b [cuda|omp|mpi|acc|sycl]

#Example
./StarPlat -s -f ../graphcode/generated_cuda/sssp_dslV2 -b omp

-s for static and -d for dynamic
-b select the type of backend
-f the dsl file to input

How to complile generated codes

SSSP

# Step 4.0 Append the main function to the generated file .cu or .cc file
# Below is the E.g. how to compile SSSP file after generation
# Additional equired files: graph.hpp libcuda.hxx present in one level up

# Step 4.1 Compile and run CUDA algorithm
nvcc -O3 -std=c++14 ../graphcode/generated_cuda/sssp_dslV2.cu -I .. -o ssspCuda.out
./ssspCuda.out ../dataset/inputfile.txt

# Step 4.2 Compile and run OMP algorithm
g++ -O3 -fopenmp -std=c++14 ../graphcode/generated_omp/sssp_dslV2.cu -I .. -o ssspOmp.out
./ssspOmp.out ../dataset/inputfile.txt

# Step 4.3 Compile and run MPI algorithm
mpcxx -fopenmp -std=c++14 ../graphcode/generated_mpi/sssp_dslV2.cu -I .. -I boost_1_74_0/ -o ssspMpi.out
./ssspMpi.out ../dataset/inputfile.txt

# Step 4.4 Compile and run ACC algorithm
pgc++ -std=c++14 ../graphcode/generated_OpenACC/sssp_dslV2.cu -I .. -o ssspAcc.out
./ssspAcc.out ../dataset/inputfile.txt

Miscellaneous

Installing Boost Libraries to run StarPlat for MPI

On the root directory of StarPlat, running the command:

grep -r '<boost/' .

shows the files that use boost. This section provides a walkthrough of building, installing, and linking the boost libraries and source to create a program. The workflow has been tested on an arm64 architecture with a 64-bit address type on a Mac machine. While this guide is compiled from various sources, for a comprehensive understanding, please refer to the official boost documentation.

Our main goal is to build the following file with mpic++ and run with mpirun:

#include <iostream>
#include <boost/mpi/environment.hpp>
#include <boost/mpi/communicator.hpp>

namespace mpi = boost::mpi;

int main() {
    mpi::environment env;
    mpi::communicator world;
    std::cout << "I am process " << world.rank() << " of " << world.size() << "." << std::endl;
    return 0;
}

The default compiler that comes with MacOS is clang, and StarPlat is built on gcc. Note that a C++ library compiled with GCC is not compatible with Clang and vice versa. So, we will build the libraries and code exclusively on one platform.

There are options to compile boost as a universal library agnostic to architecture (discussed here), but for brevity, this walkthrough aims to run it on gcc/g++.

We will be building the boost library from the source, providing more autonomy and control over packages and libraries, and installation directories from where we can include headers and link libraries.

# Compiler configuration. This definition will be used unless
# you already have defined some toolsets in your user-config.jam
# file.
# if ! clang in [ feature.values <toolset> ]
# {
#    using clang ;
# }

# project : default-build <toolset>clang ;
using gcc : <version> : /path/to/gcc
# Example
using gcc : 13.2 : /usr/local/bin/gcc
using mpi ;
export DYLD_LIBRARY_PATH=/path/to/boost/lib/:$DYLD_LIBRARY_PATH
#include <iostream>
#include <boost/mpi/environment.hpp>
#include <boost/mpi/communicator.hpp>

namespace mpi = boost::mpi;

int main() {
    mpi::environment env;
    mpi::communicator world;
    std::cout << "I am process " << world.rank() << " of " << world.size() << "." << std::endl;
    return 0;
}
OMPI_CXX=g++ mpic++ -I/Users/durwasa/boost/include -L/Users/durwasa/boost/lib helloworld.cpp -o helloworld -lboost_mpi

The OMPI_CXX sets the environment variable to gcc instead of clang (since mpic++ has been installed using Homebrew/MacPorts). -I includes the header files, -L links the library, and -lboost_mpi links the boost.mpi, which we configured using our .jam file.

Graph DSL for basic graph algorithms

Acknowledgments

This project is funded by India's National Supercomputing Mission, whose timely funding and regular reviews are greatly appreciated.

Licensing

The StarPlat codebase is made available for academic and non-commercial use. Commercial use of the code needs licensing. The interested may contact rupesh@cse.iitm.ac.in.