HDF5 Sequential Performance

aparnasasidharan2017 commented 1 year ago

Measure HDF5 Sequential Performance for different chunk sizes and file sizes.
Measure average read time and total read time for a fixed file and different chunk sizes
Measure performance on Ares

MSP23SCM92J commented 1 year ago

Benchmark_table

Bar graph: dataset(GB) vs time(second) Benchmarks_bar

line graph: chunk size vs time(second) Benchmark_line

Benchmark test results executed for different datasets and chunk size

fkengun commented 1 year ago

Comments:

Y-axis should always be bandwidth (e.g. MB/s)
Make sure benchmark is run on NVMe SSD (/mnt/nvme/username) or SATA SSD (/mnt/ssd/username) on compute node of Ares, do not run it on the master node or use /mnt/hdd/username on compute nodes
Increase event size (e.g. 1MB)
Feature support tests (e.g. how many chunks HDF5 support?)
Chronicle (c1) -> HDF5 file (c1.h5), Story (s1) -> HDF5 dataset (dataset s1 in c1.h5)
Goal: a simulator to accept time chunks of events (a bunch of events w/ Chronicle name C, Story name S, time range T1_T2), write to a dataset S in a HDF5 file C.h5 w/ dataset chunks matching time chunks

grc-iit / ChronoLog