patrickzib / SFA

Scalable Time Series Data Analytics
GNU General Public License v3.0
312 stars 67 forks source link

Non-transformed data in MFT #2

Closed seaney11 closed 7 years ago

seaney11 commented 7 years ago

In the MFT class, the fft object is initialised with the windowSize when the MFT instance is initialised, however when the array size is set later:

int arraySize = Math.max(l+this.startOffset, this.windowSize);

and when this is used in line 107:

mftData = Arrays.copyOf(timeSeries.getData(), arraySize);
this.fft.realForward(mftData);
mftData[1] = 0;

if arraySize is larger than the windowSize, then it leaves some of the mftData not transformed and is then used later on in the rest of the MFT and in the classification.

I saw this happen at windowSize = 5, l = 4 and so arraySize = 6

This has an implication on the creation of words especially at small window sizes, however I am not sure how often this occurs or how much it affects the final results

patrickzib commented 7 years ago

Wow, thanks a lot for pointing this out!

Now, the Fourier transform is always applied to data of size windowSize:

double[] dft = new double[this.windowSize];
System.arraycopy(timeSeries.getData(), 0, dft, 0, this.windowSize);
this.fft.realForward(dft);

// if windowSize > mftData.length, the remaining data should be 0 now.
System.arraycopy(dft, 0, mftData, 0, Math.min(mftData.length, dft.length));

Any remaining coefficients should be 0 from thereon.

This should fix the issue.

seaney11 commented 7 years ago

great, thanks for the response.