search

jessecusack / perturb

A tool for processing microstructure data from Rockland Scientific instruments
GNU General Public License v3.0
2 stars 1 forks source link

saving shear spectra uses too much storage #19

Open jessecusack opened 1 year ago

jessecusack commented 1 year ago

For a single 1000 m depth profile we go from a ~30 MB p file, to a ~110 MB p2mat file to a ~ 500 MB profile file... a whopping order of magnitude increase in storage requirements. 0.5 GB per profile is too much. I think that saving the p2mat file makes a lot of sense because it saves time loading the data, but we should not save all windowed shear / acceleration spectra.

jessecusack commented 1 year ago

What is the bbl structure? Why does it exist?

It looks like it contains epsilon computed bottom to top, and without the surface removed...

mousebrains commented 1 year ago

BBL is a bottom up dissipation estimate. It came from conversations with Fucent and you thinking about bottom crashing. The concept was to add bottom crash detection, then to estimate the epsilons near the bottom better, possibly with different parameters than the top down.

mousebrains commented 1 year ago

Doing a space usage audit is on my list. Some of what is there was for diagnostic purposes. We'll see what I think is droppable.