search

timeseriesAI / tsai

Time series Timeseries Deep Learning Machine Learning Python Pytorch fastai | State-of-the-art Deep Learning library for Time Series and Sequences in Pytorch / fastai
https://timeseriesai.github.io/tsai/
Apache License 2.0
5.21k stars 651 forks source link

output nan #572

Closed CHonChou closed 1 year ago

CHonChou commented 2 years ago
image

Hi, when I train the regression model using tsai, I get nan output on my own data, but when I change the arch to TSTPlus the result is normal again. Looking forward to receiving your reply!

vrodriguezf commented 2 years ago

What about InceptionTimePlus?

123mitnik commented 2 years ago

it happens to me when i set the device="cpu"
(i need to set it to cpu - because of the MPSNDArray.mm:782: failed assertion `[MPSNDArray, initWithBuffer:descriptor:] Error: buffer is not large enough. Must be 486400 bytes)

oguiza commented 2 years ago

Hi @123mitnik, I've never seen that issue before. Could you please post a short code snippet to try and reproduce this error?

123mitnik commented 2 years ago

@oguiza Hi Ignacio,

This is from pytorch. So, it has nothing to do with tsai - https://github.com/pytorch/pytorch/issues/86152

This is just because I can't run it with the nightly build of pytorch (where m1 apple silicon is supported). Tsai is not yet compatible with it (some printing issues - but I never had time to dig into it), so i'm running it on CPU. As for the @CHonChou issue - the nans may be coming from -inf values or nan inside the dataset.

oguiza commented 1 year ago

Once Pytorch and fastai fully support the new apple chips we'll make tsai compatible as well. But I'm afraid there's not much we can do for now.