search

SCIInstitute / dSpaceX

dSpaceX Visualization Library and Tool
5 stars 2 forks source link

partition data into matlab and other separate directories #73

Closed cchriste closed 4 years ago

cchriste commented 4 years ago

There are 10s of gigabytes of matlab data (.mat files) in each data directory that aren't useful outside matlab but still get copied when data is moved around [from /usr/sci/projects/dSpaceX].

This data should be stored in a separate tree from the rest. 1) matlab output should be changed to use parallel trees for the produced .mat files vs models 2) existing data should be reorganized so it can be easily used by others without copying .mat files (currently ~58G for CantileverBeam and ~122G for supershapes (nanoparticles).

cchriste commented 4 years ago

The matlab-specific portions of data have been moved, and the data has been organized into preprocessed and processed directories by latest data_processing work, now merged to master.