search

jessecusack / perturb

A tool for processing microstructure data from Rockland Scientific instruments
GNU General Public License v3.0
2 stars 1 forks source link

MR processing #14

Closed jessecusack closed 1 year ago

jessecusack commented 1 year ago

I am starting the pull request so that we can review this code update.

jessecusack commented 1 year ago

@mousebrains, I have been working off this branch to try and process the MR data locally. I just made a commit changing the flag "omitmissing" to "omitnan" in various min/median functions. The "omitmissing" flag requires matlab 2023a, which I do not have. I think that we should be careful to be inclusive of older matlab versions where possible, so long as it doesn't introduce dramatic changes. I assume that we will not encounter any missing values that are not just NaN?

mousebrains commented 1 year ago

For non-sparse double arrays, omitnan and omitmissing are equivalent. I've run across issues when using sparse arrays and omitnan. omitmissing solved my sparse array issues. I've switched to omitmissing since it "always" works, while omitnan mostly works.

We'll also have to pay attention to means of times. I may have simplified the code by using omitmissing instead testing the data type and using omitnan or omitnat.

jessecusack commented 1 year ago

I just submitted a commit reworking the CF attributes, addressing #8.

jessecusack commented 1 year ago

On you point about averaging over time I think that we should add a check to ensure there are no NaT values and drop any that exist. Why would they occur, is it possible?

Also, this code works fine for me:

>> mean([datetime("2023-01-01") NaT], "omitnan")

ans = 

  datetime

   01-Jan-2023

So maybe this isn't a problem?

mousebrains commented 1 year ago

Collapse the branch so I can start the refactorization effort.