Closed jrfackler closed 1 year ago
Hi @jrfackler,
I'm not exactly sure what you are trying to achieve with tfms = [None, [TSMaskOut(magnitude=0.0)]]
, but that's an incorrect of of a batch transform.
If your task is a classification task, you should use tfms=[None, TSClassification()]
. That way you will transform the dependent variable (y) into an int with x # classes. You can pass a vocab if you need it.
TSMaskOut is a batch transform, that is meant to be applied to the independent variables (X). Any transform with a magnitude of 0 has no effect.
Please, let me know if that solves your issue.
Hi @oguiza,
Thanks for the reply and suggestion. That piece of code was an old relic I think I used to mask out variables in a non-Classification problem but it didn't improve the results so I had set the magnitude to 0.0 and forgotten about it.
I removed that code and I still get the same error when running on version 0.3.5. If I install 0.3.4 and then 0.3.5 on top of it there is no error.
But with just 0.3.5, I get the same error.
When I google search the error, the only thing I see is that it could be related to metrics. I'm not doing an accuracy problem so I have it set to None and don't want any metrics. Could that be an issue with the new version of tsai?
This is the modified code with tfms removed
n_epochs = n_epochs
dsets = TSDatasets(X, y, splits=splits, inplace=True)
dls = TSDataLoaders.from_dsets(dsets.train, dsets.valid, splits=splits, inplace=True, bs=[96, val_size])
del dsets
gc.collect()
model = TSTPlus(dls.vars, 25, dls.len, d_model=512, d_ff=512, n_layers=n_layers,
n_heads=n_heads, dropout=dropout, pe=pe, fc_dropout=fc_dropout, attn_dropout=0.0)
learn = Learner(dls, model, loss_func=CustomLoss1(), metrics=None, opt_func=SGD, cbs=ShowGraphCallback2())
start = time.time()
with ContextManagers([learn.no_logging(), learn.no_bar()]):
learn.fit_one_cycle(n_epochs, lr_max=lr_max)
Hi @jrfackler,
I've tried to reproduce the error but failed.
I'm not sure what the task it: classification, regression, forecasting, ... I don't know what X and y dtypes. But you are not using any transform.
Can you please use check_data(X, y, splits, show_plot=False)
and share the output?
Hi @oguiza,
This the output from check_data(X, y, splits, show_plot=False)
:
X - shape: [99 samples x 300 features x 35 timesteps] type: ndarray dtype:float64 isnan: 0
y - shape: (99, 25) type: ndarray dtype:float64 isnan: 0
splits - n_splits: 2 shape: [79, 20] overlap: False
I'm attaching a link to X and y datasets (doesn't seem like I can attach the datasets directly to this comment) and the code is below to reproduce the error if that helps. It works fine and completes with 0.3.4 but doesn't with 0.3.5.
Thank you very much for looking at this.
#!pip install tsai==0.3.4
!pip install tsai==0.3.5
from tsai.all import *
import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
import gc
%matplotlib inline
my_setup()
X_dataset = np.load('/notebooks/X_dataset_test.npy')
Y_dataset = np.load('/notebooks/y_dataset_test.npy')
print(X_dataset.shape)
samples = X_dataset.shape[0]
n_split = round(0.8 * samples)
X_train = X_dataset[0:n_split, :, :]
y_train = Y_dataset[0:n_split]
X_valid = X_dataset[n_split:, :, :]
y_valid = Y_dataset[n_split:]
print(X_dataset.shape)
print(Y_dataset.shape)
print(X_train.shape)
print(y_train.shape)
print(X_valid.shape)
print(y_valid.shape)
X, y, splits = combine_split_data([X_train, X_valid], [y_train, y_valid])
class CustomLoss1(nn.Module):
def __init__(self):
super().__init__()
def forward(self, input: Tensor, target: Tensor) -> Tensor:
abs_input = torch.abs(input)
sum_input = torch.sum(abs_input, dim=1, keepdim=True)
scaled_input = torch.div(input, sum_input)
values = torch.mul(scaled_input, target)
dp = torch.sum(values, dim=1)
adp = torch.mean(dp)
neg_days = torch.lt(dp, 0.0)
np = torch.mul(dp, neg_days)
nps = torch.square(np)
npa = torch.mean(nps)
neg_stdev = torch.sqrt(npa)
neg_stdev_percent = torch.mul(neg_stdev, 100.0)
adp_plus1 = torch.add(adp, 1.0)
yearly_total = torch.pow(adp_plus1, 252.0)
yp = torch.sub(yearly_total, 1.0)
yearly_percent = torch.mul(yp, -100.0)
yearly_neg_stdev = torch.pow(torch.mul(neg_stdev_percent, 19.105), 1.0)
loss = torch.add(yearly_percent, yearly_neg_stdev)
return loss
n_epochs = 10
n_layers = 2
n_heads = 8
dropout = 0.3
fc_dropout = 0.8
lr_max = 3.5e-7
pe = 'sincos'
val_size = X_valid.shape[0]
n_epochs = n_epochs
dsets = TSDatasets(X, y, splits=splits, inplace=True)
dls = TSDataLoaders.from_dsets(dsets.train, dsets.valid, splits=splits, inplace=True, bs=[96, val_size])
model = TSTPlus(dls.vars, 25, dls.len, d_model=512, d_ff=512, n_layers=n_layers,
n_heads=n_heads, dropout=dropout, pe=pe, fc_dropout=fc_dropout, attn_dropout=0.0)
learn = Learner(dls, model, loss_func=CustomLoss1(), metrics=None, opt_func=SGD, cbs=ShowGraphCallback2())
start = time.time()
check_data(X, y, splits, show_plot=False)
with ContextManagers([learn.no_logging(), learn.no_bar()]):
learn.fit_one_cycle(n_epochs, lr_max=lr_max)
Hi @jrfackler, You are using a classification task with 25 classes, but y is a float. That is an issue. You need to convert y to int. That is handled by a tfm (TSClassification()). You can adapt (and simplify) your code using something like:
tfms = [None, TSClassification()]
arch_config = dict(d_model=512, d_ff=512, n_layers=n_layers,
n_heads=n_heads, dropout=dropout, pe=pe, fc_dropout=fc_dropout, attn_dropout=0.0)
learn = TSClassifier(X, y, splits=splits, tfms=tfms, arch="TSTPlus", arch_config=arch_config,
loss_func=CustomLoss1(), opt_func=SGD, metrics=None, cbs=ShowGraph())
with ContextManagers([learn.no_logging(), learn.no_bar()]):
learn.fit_one_cycle(n_epochs, lr_max=lr_max)
Hi @oguiza,
Thank you. Your suggestion helped fix my training issue on the new version.
While not a classification task, I used your code and swapped out TSRegressor for TSClassifier and the training worked. I had been using code from a couple years ago I took from a tutorial notebook but I see that updates to tsai now use a different format.
One interesting thing about using TSRegressor and the new version of tsai is that the hyperparameters needed to get similar results on the validation set need to be adjusted when compared to my previous code and the old version of tsai.
This created one more issue though that I can't find an answer to. When doing inference using the code:
_, _, input = learn.get_X_preds(X_valid, with_decoded=True)
The variable input
is now a: fastai.torch_core.TitledTuple
when I think before it was just a Tensor. And when I try to use torch functions on it I get errors that it needs to be a Tensor.
Do you know how to covert it back to a Tensor? - I realized I guess I can just use the torch.tensor(tuple)
to convert it so no more questions. Thanks for your help!
I'll close this issue based on feedback received.
Hello,
Any anyone seen this error with the current version of tsai?
This error only occurs when using version 0.3.5. The previous version 0.3.4 does not produce the error. A quick google search shows that this may be related to metrics but I have metrics=None I'm using a custom loss function
n_epochs = n_epochs tfms = [None, [TSMaskOut(magnitude=0.0)]] dsets = TSDatasets(X, y, tfms=tfms, splits=splits, inplace=True) dls = TSDataLoaders.from_dsets(dsets.train, dsets.valid, splits=splits, tfms=tfms, inplace=True, bs=[96, val_size], batch_tfms=None) del dsets gc.collect()
model = TSTPlus(dls.vars, 25, dls.len, d_model=512, d_ff=512, n_layers=n_layers, n_heads=n_heads, dropout=dropout, pe=pe, fc_dropout=fc_dropout, attn_dropout=0.0)
learn = Learner(dls, model, loss_func=CustomLoss1(), metrics=None, opt_func=SGD, cbs=ShowGraphCallback2())
start = time.time()
with ContextManagers([learn.no_logging(), learn.no_bar()]): learn.fit_one_cycle(n_epochs, lr_max=lr_max)
This is the error stack: