ludwig-ai / ludwig

Low-code framework for building custom LLMs, neural networks, and other AI models
http://ludwig.ai
Apache License 2.0
11.18k stars 1.19k forks source link

Setting sequence limit with timeseries input #1076

Closed mangleddata closed 3 weeks ago

mangleddata commented 3 years ago

I am attempting to use LSTM encoder with timeseries data. Is there a way to preprocess to set a limit on the sequence ? I see sequence_length_limit parameter - but not sure if this would achieve that. So the use case is, my input training file may have a timeseries input col which has a sequence of 16 values, but if I want to trim it to say 4, I would want the trailing 4 values to be used. Is that possible ? Trying to see if I can avoid recreating training file with different sequence limit.

jimthompson5802 commented 3 years ago

@mangleddata Thank you for submitting your question.

First, in researching your question, I found the Ludwig documentation requires an update related to time series. To limit the number of values returned from a time series the correct preprocessing parameter is one called timeseries_length_limit, i.e., timeseries_length_limit: 4. Use of sequence_length_limit is not applicable to a time series feature.

Second, regarding the returning trailing values from the time series. This does not work at this time. We consider is an issue and will work to fix it.

This partially answers your question. Please monitor this issue for status updates.