Virtual Quake is a boundary element code designed to investigate long term fault system behavior and interactions between faults through stress transfer.
Currently, the QuakeLib library reads in the HDF5 event/simulation files completely into memory, regardless of size. Sometimes we have simulations that reach > 10GB and doing any simple PyVQ plots requires like 20GB of RAM and takes forever.
We need: An improved model that uses, say the h5py Python module to incrementally read in the event file. Maybe we should just be reading in 1000 events at a time, computing whatever we need, then keep the plot-able results and delete the events from memory ("del" in Python).
Currently, the QuakeLib library reads in the HDF5 event/simulation files completely into memory, regardless of size. Sometimes we have simulations that reach > 10GB and doing any simple PyVQ plots requires like 20GB of RAM and takes forever.
We need: An improved model that uses, say the h5py Python module to incrementally read in the event file. Maybe we should just be reading in 1000 events at a time, computing whatever we need, then keep the plot-able results and delete the events from memory ("del" in Python).