Using augmentation as standalone

[Willtl]

Is there a way that I can use your augmentation as a standalone within my torch dataset?

I'm used to using timm for images. I hope there is some way of using tsai augmentations as there is on timm.

If someone can provide an example (or point me towards it) of using a simple Resize, or RandomCrop on a numpy array, it would greatly help.

[oguiza]

Hi @Willtl, I'm not exactly sure what you mean by "using augmentation as standalone", and I'm not familiar with how timm uses augmentations. To learn how augmentations work in tsai, you can take a look at:

• docs
• tutorial nb: you'll be able to see almost all augmentations available in tsai.

Please, let me know if this answers your question or you are looking for something different.

[Willtl]

Thanks for sharing it.

I just want to use like this: augmented = TSTimeNoise(magnitude=10)(series) But the augmented version looks exactly the same as the series. It seems that the encode is not being called. I also tried like this: augmented = TSTimeNoise(magnitude=10).encodes(TSTensor(series)), and it also does not work.

[oguiza]

Ah, ok. I understand. The issue is that you are not passing the split_idx=0 to let the transform know that the data is part of the training set (the validation set is not transformed by default, unless you specifically want it that way). You can try this code:

series = TSTensor(np.arange(100)).float()
augmented = TSTimeNoise()(TSTensor(series), split_idx=0)
print(augmented.data)

When you want to transform the validation set, you need to use:

series = TSTensor(np.arange(100)).float()
augmented = TSTimeNoise(split_idx=1)(TSTensor(series), split_idx=1)
print(augmented.data)

The split_idx that is passed with the data is automatically added by the dataloader.

[Willtl]

Hi @oguiza, thank you for your help earlier. I was able to get everything working properly now.

I do have a question about using TSTensor() from the tsai library. As I am only using the transforms from tsai, I'm wondering what advantages there might be to using TSTensor() instead of Torch tensors.

It appears that the transforms are not compatible with numpy arrays or Torch tensors directly, so I would need to convert them to TSTensor() first. Can you confirm if my understanding is correct?

Thanks!

[oguiza]

Yes, that's correct. The reason is that in tsai a dataloader may create different types of tensors. And with these type of transforms you'll only modify those that are of type TSTensor. I'm not sure what's your use case is and how you are using these transforms, but it won't take much time nor memory to apply TSTensor to a numpy array. You can check this using:

a = np.random.rand(10).astype('float32')
t = TSTensor(a)
np.shares_memory(a, t)
# True

So while I understand it may not be necessary in your case, I don't see any major issue with this approach. Agree?

[Willtl]

I was able to use the tsai augmentation successfully during training, but I noticed that it significantly increased my training time. Without tsai augmentation, my mean epoch time was 0.23 seconds. However, when I added just one augmentation (TSRandomResizedCrop), the mean epoch time increased to 0.98 seconds. I'm not sure if this is expected or if there's something I'm doing wrong.

I'm used to working with image augmentations using torchvision transformations, and I know there's typically some overhead associated with using augmentations, but this increase in training time seems like a lot.

Here is how I implemented the tsai resize crop aug. on my custom torch dataset:

    def __getitem__(self, index):
        serie = self.raw[index] # raw TSTensor of the time serie
        shape = serie.shape # here shape is (6, 128)
        serie = serie.view(1, shape[0], shape[1]) # view it (1, 6, 128), so I can use tsai augmentation
        aug = TSRandomResizedCrop(scale=(0.1, 1.0))(serie, split_idx=0).view(shape[0], shape[1])
        # I'm not computing the updated fft of the augmentation so I can check how much tsai aug. will increase in time
        # aug_fft = torch.from_numpy(np.abs(np.fft.rfft(aug))).float()
        return aug, self.fft[index], self.bin_target[index], self.target[index]

[oguiza]

Please, take a look at this gist to see the type of time delta you should get. The main issue I see is that you are instantiating TSRandomResizedCrop every time you are calling getitem. You should instantiate it in init (something like self.tfm= TSRandomResizedCrop(scale=(0.1, 1.0)) and then apply it in getitem). BTW, there's a torch.fft.rfft function that can be run on the GPU.

[Willtl]

Indeed, instantiation TSRandomResizedCrop reduced mean epoch time to 0.64.

Then decided to derive from TSRandomResizedCrop encode and created my version with fixed linear interpolation, like:

def crop_resize(scale, size, o: TSTensor):
    seq_len = o.shape[-1]
    lambd = np.random.uniform(scale[0], scale[1])
    win_len = int(round(seq_len * lambd))
    if win_len == seq_len:
        if size == seq_len: return o
        _slice = slice(None)
    else:
        start = np.random.randint(0, seq_len - win_len)
        _slice = slice(start, start + win_len)
    return F.interpolate(o[..., _slice], size=size, mode='linear', align_corners=False)

By doing so, the mean decreased from 0.64 sec. to 0.54 sec.

Lastly, with the given implemented crop_resize function, it was not required the tensors to be TSTensor; therefore, I made them torch.tensors which led the time back to 0.26 sec. (closer to the 0.23 without augmentation).

In your gists it seems it does not change much so definitely there is something wrong on my code while using TSTensors.

[oguiza]

It's known that torch.Tensor subclasses like TSTensor create a delay. There's a Pytorch bug logged about it. However, it's relatively small. You can see it in this new gist. The timings that I'm getting in Colab are mush faster than the ones you get. How big is your batch size? I've used 64 and in all cases the transform takes <1ms on a GPU and <3ms on a CPU, while it's taking you >200ms.

[Willtl]

Thanks for the help @oguiza.

One last question, do you guys benchmark your available models (or provide any metric like reported accuracy, etc)? e.g., https://github.com/huggingface/pytorch-image-models/blob/main/results/results-imagenet.csv

[oguiza]

No, we don’t. Unlike computer vision:

• there are not clearly established benchmarks in time series.
• public datasets tend to be small.

And we don’t have the resources to do it :)

[Willtl]

That is very sad. I hope I can contribute to changing this paradigm.

Many thanks so much for all the help. I'll close this.