search

petteriTeikari / R-PLR

Once separate repos combined to one making it easier in the end
1 stars 0 forks source link

[Reconstruction] EMD Inspection #1

Open petteriTeikari opened 6 years ago

petteriTeikari commented 6 years ago

Good quality recordings

1095 1095

1104_typicalcontrol 1104

Artifact removal working

1103_artifactspike_removedok 1103

1116_artifactspike_removedok 1116

4092_artifactspike_removedok 4092

4095_artifactspike_removedok 4095

Even when the constriction amplitude is small with high noise

4098_decompositionok_whilelowamplitudeandnoisy 4098

And with funky baseline

4102_decompositionok_whilefunkybaselines 4102

Nice handling of missing chunks as well

4121_missingchunk_handledok 4121

4135_missingchunk_handledok 4135

Even with an artifact spike

4159_missingchunk_and_spike_handledok 4159

But then of course some traces are Problematic: Artifact spikes still left in the signal (after removing noise in IMF1-4):

1096_artifactspike_stillleft 1096

1146_artifactspike_stillleft 1146

4077_artifactspike_stillleft 4077

4080_artifactspike_stillleft 4080

4139_artifactspike_stillleft 4139

4143_artifactspike_stillleft 4143

4148_artifactspike_stillleft 4148

4156_artifactspike_stillleft 4156

4168_artifactspike_stillleft 4168

Questionable Seems like a reconstriction or is it just an artifact?

2091_reconstriction_orartifact 2091

petteriTeikari commented 6 years ago

After running the EMD the second time after further cleaning some of the spikes and imputing the data with missForest that seems to produce better imputations than the previously used fast StructTS Kalman filter from imputeTS, the denoising quality starts to seem really good:

1005_goodquality_gettingsomeclicksremoved 1005

1010_goodquality_gettingclickremoved 1010

Denoising performs rather well under heavy noise as well

4168_removesheavynoisequitewell 4168

4211_intrinsicallynoisy 4211

And in some cases, one can inspect whether too much was taken from the extremities (in this case from max constriction), or was the denoising reasonable in the end:

1085_goodquality_withoutliermax 1085

4137_goodquality_withoutliermax 4137

In some cases, the small outlier "constrictions" could be constrictions, or just most likely outliers, but these were kept as they looked sort of "physiological", but could be removed later if wanted

4014_microconstrictionoroutlier 4014

4046_microconstrictionoroutlier 4046

4084_microconstrictionoroutlier 4084

4114_microconstrictionoroutlier 4114

missForest imputation with CEEMD handles better longer missing chunks than the simple Kalman filtering based

4121_missingchunk_withmissforest_handledok 4121, missForest + CEEMD

4121_missingchunk_handledok 4121, Kalman + CEEMD

For two cases, the missForest left some outlier spikes but they were removed on second run of missForest by manually marking those spikes as NAs and keeping the imputed values from previous runs.

See the difference for file 2056

2056_spikeleft_cleanedratherwell before

2056_rerunmissforest_withcorrection_becameok after

And for file 4148

4148_spikeleft_cleanedratherwell before

4148_rerunmissforest_withcorrection_becameok after

And the oscillations themselves might be a bit biometric feature as well?

Some subject show stronger oscillations (hiFreq) from blue stimulus:

1149_seeoscillationdifference_betweenredandblue 1149

Some from red stimulus

1172_seeoscillationdifference_betweenredandblue 1172

And some people seem to have more or less constant oscillations throughtout the recording

1174_constantoscillations_afteronsets

And some people have rather identical oscillations on light onsets:

2049_identicalishoscillations_afteronsets 2049

2050_identicalishoscillations_afteronsets 2050

And one person had very pronounced light-dependent oscillations 4177_verylightdependentoscillations

And as technical notes, when the decomposition is "perfect" with no residue left, the last IMF becomes a noise component as well:

1064_goodquality_withnoresidue_lastoneisnoise 1064

And sometimes you get three "base IMFs" as typically I only have selected the residue and the last IMF to the "base"

1153_threetimesbase