tomcucinotta / distwalk

Distributed processing emulation tool
GNU General Public License v3.0
2 stars 4 forks source link

Support multiple storage devices #60

Open tomcucinotta opened 1 week ago

tomcucinotta commented 1 week ago

For nodes with multiple storage devices, dw_node should support instantiating a load/store file on any of them, probably through a dedicated I/O thread per-device. This might be supported through multiple uses of the -s|--storage= option, specifying multiple paths belonging to multiple devices (dw_node would probably not check where the files reside). I'm unsure about whether to support, from the client-side, a specification of the disk number, corresponding to the specified storage paths in their order, or allowing for filtering via disk type (HDD vs SSD), to be added to store_opts.

For example:

dw_node -s /mnt/hdd/dw_store.dat -s /mnt/ssd/dw_store.dat
dw_client -S 10KB       # store on first device (default, a HDD in this case)
dw_client -S 10KB,dev=0 # select first device (HDD for this node)
dw_client -S 10KB,dev=1 # select second device (SSD for this node)
dw_client -S 10KB,ssd=0 # select first SSD device (second specified via -s, for this node)
dw_client -S 10KB,hdd=0 # select first HDD device available on node (same as dev=0, for this node)
deRemo commented 1 week ago

Follow-up of Issue #27