OpenMPI and OpenMP

[cindytsai]

Whether or not define OpenMP Threads Number in Code

Volume Rendering Test Run

• Post-Processing
  • [x] OpenMP + OpenMPI

    Reference

• Volume Rendering Parallel
• Volume Rendering in Hybrid

[cindytsai]

Volume Rendering in Inline-Analysis

• Test Problem: gamer 3D blastwave
• Machine: calab912
• Set OpenMP thread number: Set OMP_NTHREAD in Input_Parameter and also

  export OMP_NUM_THREADS=4 

• Inline Script:

  
  import yt
  import time
  from mpi4py import MPI

comm = MPI.COMM_WORLD myrank = comm.Get_rank()

yt.enable_parallelism() i = 0

def yt_inline(): t_start = time.time()

ds = yt.frontends.libyt.libytDataset()
# Volume Rendering
sc = yt.create_scene(ds, lens_type="perspective")
source = sc[0]
source.tfh.set_log(True)
source.tfh.grey_opacity = False
source.tfh.plot("transfer_function.png", profile_field=("gas", "density"))
global i
sc.save("rendering_" + str(i) + ".png", sigma_clip=4.0)
i += 1

t_end = time.time()
with open(str(myrank) + ".txt", "a") as file:
    file.write(str(t_end - t_start) + "\n")

### Results
 * [Link to google sheet](https://docs.google.com/spreadsheets/d/1xE4uY92KPbfW0-wgsj1rZU4-2qFNXIoLtjfA52Wnytk/edit?usp=sharing)
 * MPI = 2 with different OMP threads. _( y-axis is in time unit second, which is the time spent in volume rendering in each step.)_
    ![MPI=2](https://user-images.githubusercontent.com/6188366/144742086-9881d6da-b550-46ba-acd6-d123a97e7e43.png) 
 * OMP = 1 with different MPI sizes. _( y-axis is in time unit second, which is the time spent in volume rendering in each step.)_
    ![OMP=1](https://user-images.githubusercontent.com/6188366/144742793-9af7e77f-8721-42cb-9567-85aa2e05840a.png)

[cindytsai]

Volume Rendering in Post-Processing

• Test Problem: gamer 3D blastwave
• Machine: calab912
• Set OMP thread number:

  export OMP_NUM_THREADS=4

• Post Script:

  
  import yt
  import time
  from mpi4py import MPI

comm = MPI.COMM_WORLD myrank = comm.Get_rank()

yt.enable_parallelism() i = 0

def yt_inline(): t_start = time.time()

ds = yt.load("Data_000016")
# Volume Rendering
sc = yt.create_scene(ds, lens_type="perspective")
source = sc[0]
source.tfh.set_log(True)
source.tfh.grey_opacity = False
source.tfh.plot("transfer_function.png", profile_field=("gas", "density"))
global i
sc.save("rendering_" + str(i) + ".png", sigma_clip=4.0)
i += 1

t_end = time.time()
with open(str(myrank) + ".txt", "a") as file:
    file.write(str(t_end - t_start) + "\n")
    print("Rank=", myrank, t_end - t_start)

yt_inline()

### Result
Time spent in step16:
| OMP=1 | OMP=2 | OMP=4 | OMP=8 |
|:-----:|:-----:|:-----:|:-----:|
|  8.04 |  7.39 |  6.99 |  6.99 |

- Scaling is better than in inline-analysis, but is still bad.