search

LLNL / MACSio

A Multi-purpose, Application-Centric, Scalable I/O Proxy Application
https://computing.llnl.gov/projects/co-design/macsio
Other
34 stars 16 forks source link

Add CL args to control file system dir naming #27

Open HenryYihengXu opened 4 years ago

HenryYihengXu commented 4 years ago

Macsio generates a large number of log files. Is it possible to redirect those files to another directory in the command line?

login-1:macsio_darshan$ ls
darshan_runtime_logs        macsio_silo_00002_005.silo  macsio_silo_00005_004.silo
macsio              macsio_silo_00002_006.silo  macsio_silo_00005_005.silo
macsio-log.log          macsio_silo_00002_007.silo  macsio_silo_00005_006.silo
macsio-timings.log      macsio_silo_00002_008.silo  macsio_silo_00005_007.silo
macsio_silo_00000_000.silo  macsio_silo_00002_009.silo  macsio_silo_00005_008.silo
macsio_silo_00000_001.silo  macsio_silo_00003_000.silo  macsio_silo_00005_009.silo
macsio_silo_00000_002.silo  macsio_silo_00003_001.silo  macsio_silo_00006_000.silo
macsio_silo_00000_003.silo  macsio_silo_00003_002.silo  macsio_silo_00006_001.silo
macsio_silo_00000_004.silo  macsio_silo_00003_003.silo  macsio_silo_00006_002.silo
macsio_silo_00000_005.silo  macsio_silo_00003_004.silo  macsio_silo_00006_003.silo
macsio_silo_00000_006.silo  macsio_silo_00003_005.silo  macsio_silo_00006_004.silo
macsio_silo_00000_007.silo  macsio_silo_00003_006.silo  macsio_silo_00006_005.silo
macsio_silo_00000_008.silo  macsio_silo_00003_007.silo  macsio_silo_00006_006.silo
macsio_silo_00000_009.silo  macsio_silo_00003_008.silo  macsio_silo_00006_007.silo
macsio_silo_00001_000.silo  macsio_silo_00003_009.silo  macsio_silo_00006_008.silo
macsio_silo_00001_001.silo  macsio_silo_00004_000.silo  macsio_silo_00006_009.silo
markcmiller86 commented 4 years ago

Hmm....it is designed to be very frugal w.r.t. to log files and I see only two from macsio itself there macsio-log.log and macsio-timings.log. All the other .silo files are data files probably run in file-per-processor mode. Changing command-line arguments regarding -parallel_file_mode MIF affects this. Or, for those plugins that support single-shared-file like HDF5 plugin, using -parallel_file_mode SIF.

Am I miss-understanding your question?

HenryYihengXu commented 4 years ago

Sorry for the unclear part. What I wanted to say are those .silo files. By command-line arguments, is there a way to put them into another directory instead of the current directory? They make the current directory kinda messy

markcmiller86 commented 4 years ago

Ok, yeah, I get you. Currently, we don't support that. We should probably operate that way by default and maybe provide a CL arg. to override it.

markcmiller86 commented 4 years ago

I've modified the issue heading to reflect the desired enhancement.

HenryYihengXu commented 3 years ago

Hi just want to check how's this issue going now. I want to run Macsio with Burst Buffers on Summit. I can write or read files in the burst buffer directory, but I cannot directly cd into the burst buffer directory. Therefore, if I want macsio to do I/O in that directory, I need to pass the directory to Macsio.