search

Nixtla / nixtla

TimeGPT-1: production ready pre-trained Time Series Foundation Model for forecasting and anomaly detection. Generative pretrained transformer for time series trained on over 100B data points. It's capable of accurately predicting various domains such as retail, electricity, finance, and IoT with just a few lines of code 🚀.
https://docs.nixtla.io
Other
2.31k stars 187 forks source link

chore: add efficiency and one billion series experiments #463

Closed jmoralez closed 2 months ago

github-actions[bot] commented 2 months ago
Experiment Results ## Experiment 1: air-passengers ### Description: | variable | experiment | |:--------------|:-------------| | h | 12 | | season_length | 12 | | freq | MS | | level | None | | n_windows | 1 | ### Results: | metric | timegpt-1 | timegpt-1-long-horizon | SeasonalNaive | Naive | |:-----------|------------:|-------------------------:|----------------:|-----------:| | mae | 12.6793 | 11.0623 | 47.8333 | 76 | | mape | 0.027 | 0.0232 | 0.0999 | 0.1425 | | mse | 213.936 | 199.132 | 2571.33 | 10604.2 | | total_time | 0.9929 | 0.6793 | 0.0054 | 0.004 | ### Plot: ![](https://github.com/Nixtla/nixtla/blob/docs-figs-model-performance//action_files/models_performance/plots/plot_air-passengers_12_12_MS_None_1.png?raw=true) ## Experiment 2: air-passengers ### Description: | variable | experiment | |:--------------|:-------------| | h | 24 | | season_length | 12 | | freq | MS | | level | None | | n_windows | 1 | ### Results: | metric | timegpt-1 | timegpt-1-long-horizon | SeasonalNaive | Naive | |:-----------|------------:|-------------------------:|----------------:|-----------:| | mae | 58.1031 | 58.4587 | 71.25 | 115.25 | | mape | 0.1257 | 0.1267 | 0.1552 | 0.2358 | | mse | 4040.22 | 4110.79 | 5928.17 | 18859.2 | | total_time | 0.9284 | 0.5662 | 0.0044 | 0.004 | ### Plot: ![](https://github.com/Nixtla/nixtla/blob/docs-figs-model-performance//action_files/models_performance/plots/plot_air-passengers_24_12_MS_None_1.png?raw=true) ## Experiment 3: electricity-multiple-series ### Description: | variable | experiment | |:--------------|:-------------| | h | 24 | | season_length | 24 | | freq | H | | level | None | | n_windows | 1 | ### Results: | metric | timegpt-1 | timegpt-1-long-horizon | SeasonalNaive | Naive | |:-----------|------------:|-------------------------:|----------------:|---------------:| | mae | 178.293 | 268.121 | 269.23 | 1331.02 | | mape | 0.0234 | 0.0311 | 0.0304 | 0.1692 | | mse | 121588 | 219457 | 213677 | 4.68961e+06 | | total_time | 0.5959 | 3.2593 | 0.0056 | 0.0051 | ### Plot: ![](https://github.com/Nixtla/nixtla/blob/docs-figs-model-performance//action_files/models_performance/plots/plot_electricity-multiple-series_24_24_H_None_1.png?raw=true) ## Experiment 4: electricity-multiple-series ### Description: | variable | experiment | |:--------------|:-------------| | h | 168 | | season_length | 24 | | freq | H | | level | None | | n_windows | 1 | ### Results: | metric | timegpt-1 | timegpt-1-long-horizon | SeasonalNaive | Naive | |:-----------|------------:|-------------------------:|----------------:|---------------:| | mae | 465.532 | 346.984 | 398.956 | 1119.26 | | mape | 0.062 | 0.0437 | 0.0512 | 0.1583 | | mse | 835120 | 403787 | 656723 | 3.17316e+06 | | total_time | 0.6701 | 0.7175 | 0.0057 | 0.0051 | ### Plot: ![](https://github.com/Nixtla/nixtla/blob/docs-figs-model-performance//action_files/models_performance/plots/plot_electricity-multiple-series_168_24_H_None_1.png?raw=true) ## Experiment 5: electricity-multiple-series ### Description: | variable | experiment | |:--------------|:-------------| | h | 336 | | season_length | 24 | | freq | H | | level | None | | n_windows | 1 | ### Results: | metric | timegpt-1 | timegpt-1-long-horizon | SeasonalNaive | Naive | |:-----------|--------------:|-------------------------:|----------------:|---------------:| | mae | 558.649 | 459.769 | 602.926 | 1340.95 | | mape | 0.0697 | 0.0566 | 0.0787 | 0.17 | | mse | 1.22721e+06 | 739135 | 1.61572e+06 | 6.04619e+06 | | total_time | 0.9486 | 0.8791 | 0.006 | 0.0054 | ### Plot: ![](https://github.com/Nixtla/nixtla/blob/docs-figs-model-performance//action_files/models_performance/plots/plot_electricity-multiple-series_336_24_H_None_1.png?raw=true)
github-actions[bot] commented 2 months ago
Experiment Results ## Experiment 1: air-passengers ### Description: | variable | experiment | |:--------------|:-------------| | h | 12 | | season_length | 12 | | freq | MS | | level | None | | n_windows | 1 | ### Results: | metric | timegpt-1 | timegpt-1-long-horizon | SeasonalNaive | Naive | |:-----------|------------:|-------------------------:|----------------:|-----------:| | mae | 12.6793 | 11.0623 | 47.8333 | 76 | | mape | 0.027 | 0.0232 | 0.0999 | 0.1425 | | mse | 213.936 | 199.132 | 2571.33 | 10604.2 | | total_time | 1.194 | 0.7597 | 0.0074 | 0.0044 | ### Plot: ![](https://github.com/Nixtla/nixtla/blob/docs-figs-model-performance//action_files/models_performance/plots/plot_air-passengers_12_12_MS_None_1.png?raw=true) ## Experiment 2: air-passengers ### Description: | variable | experiment | |:--------------|:-------------| | h | 24 | | season_length | 12 | | freq | MS | | level | None | | n_windows | 1 | ### Results: | metric | timegpt-1 | timegpt-1-long-horizon | SeasonalNaive | Naive | |:-----------|------------:|-------------------------:|----------------:|-----------:| | mae | 58.1031 | 58.4587 | 71.25 | 115.25 | | mape | 0.1257 | 0.1267 | 0.1552 | 0.2358 | | mse | 4040.21 | 4110.79 | 5928.17 | 18859.2 | | total_time | 1.0274 | 0.9003 | 0.0047 | 0.0045 | ### Plot: ![](https://github.com/Nixtla/nixtla/blob/docs-figs-model-performance//action_files/models_performance/plots/plot_air-passengers_24_12_MS_None_1.png?raw=true) ## Experiment 3: electricity-multiple-series ### Description: | variable | experiment | |:--------------|:-------------| | h | 24 | | season_length | 24 | | freq | H | | level | None | | n_windows | 1 | ### Results: | metric | timegpt-1 | timegpt-1-long-horizon | SeasonalNaive | Naive | |:-----------|------------:|-------------------------:|----------------:|---------------:| | mae | 178.293 | 268.121 | 269.23 | 1331.02 | | mape | 0.0234 | 0.0311 | 0.0304 | 0.1692 | | mse | 121588 | 219457 | 213677 | 4.68961e+06 | | total_time | 0.7391 | 1.8491 | 0.0053 | 0.0052 | ### Plot: ![](https://github.com/Nixtla/nixtla/blob/docs-figs-model-performance//action_files/models_performance/plots/plot_electricity-multiple-series_24_24_H_None_1.png?raw=true) ## Experiment 4: electricity-multiple-series ### Description: | variable | experiment | |:--------------|:-------------| | h | 168 | | season_length | 24 | | freq | H | | level | None | | n_windows | 1 | ### Results: | metric | timegpt-1 | timegpt-1-long-horizon | SeasonalNaive | Naive | |:-----------|------------:|-------------------------:|----------------:|---------------:| | mae | 465.532 | 346.984 | 398.956 | 1119.26 | | mape | 0.062 | 0.0437 | 0.0512 | 0.1583 | | mse | 835120 | 403787 | 656723 | 3.17316e+06 | | total_time | 1.0554 | 2.5534 | 0.0056 | 0.0057 | ### Plot: ![](https://github.com/Nixtla/nixtla/blob/docs-figs-model-performance//action_files/models_performance/plots/plot_electricity-multiple-series_168_24_H_None_1.png?raw=true) ## Experiment 5: electricity-multiple-series ### Description: | variable | experiment | |:--------------|:-------------| | h | 336 | | season_length | 24 | | freq | H | | level | None | | n_windows | 1 | ### Results: | metric | timegpt-1 | timegpt-1-long-horizon | SeasonalNaive | Naive | |:-----------|--------------:|-------------------------:|----------------:|---------------:| | mae | 558.649 | 459.769 | 602.926 | 1340.95 | | mape | 0.0697 | 0.0566 | 0.0787 | 0.17 | | mse | 1.22721e+06 | 739135 | 1.61572e+06 | 6.04619e+06 | | total_time | 1.368 | 1.9365 | 0.0057 | 0.0068 | ### Plot: ![](https://github.com/Nixtla/nixtla/blob/docs-figs-model-performance//action_files/models_performance/plots/plot_electricity-multiple-series_336_24_H_None_1.png?raw=true)
github-actions[bot] commented 2 months ago
Experiment Results ## Experiment 1: air-passengers ### Description: | variable | experiment | |:--------------|:-------------| | h | 12 | | season_length | 12 | | freq | MS | | level | None | | n_windows | 1 | ### Results: | metric | timegpt-1 | timegpt-1-long-horizon | SeasonalNaive | Naive | |:-----------|------------:|-------------------------:|----------------:|-----------:| | mae | 12.6793 | 11.0623 | 47.8333 | 76 | | mape | 0.027 | 0.0232 | 0.0999 | 0.1425 | | mse | 213.936 | 199.132 | 2571.33 | 10604.2 | | total_time | 0.6184 | 0.7996 | 0.0059 | 0.0038 | ### Plot: ![](https://github.com/Nixtla/nixtla/blob/docs-figs-model-performance//action_files/models_performance/plots/plot_air-passengers_12_12_MS_None_1.png?raw=true) ## Experiment 2: air-passengers ### Description: | variable | experiment | |:--------------|:-------------| | h | 24 | | season_length | 12 | | freq | MS | | level | None | | n_windows | 1 | ### Results: | metric | timegpt-1 | timegpt-1-long-horizon | SeasonalNaive | Naive | |:-----------|------------:|-------------------------:|----------------:|-----------:| | mae | 58.1031 | 58.4587 | 71.25 | 115.25 | | mape | 0.1257 | 0.1267 | 0.1552 | 0.2358 | | mse | 4040.21 | 4110.79 | 5928.17 | 18859.2 | | total_time | 0.6013 | 0.4889 | 0.0045 | 0.004 | ### Plot: ![](https://github.com/Nixtla/nixtla/blob/docs-figs-model-performance//action_files/models_performance/plots/plot_air-passengers_24_12_MS_None_1.png?raw=true) ## Experiment 3: electricity-multiple-series ### Description: | variable | experiment | |:--------------|:-------------| | h | 24 | | season_length | 24 | | freq | H | | level | None | | n_windows | 1 | ### Results: | metric | timegpt-1 | timegpt-1-long-horizon | SeasonalNaive | Naive | |:-----------|------------:|-------------------------:|----------------:|---------------:| | mae | 178.293 | 268.121 | 269.23 | 1331.02 | | mape | 0.0234 | 0.0311 | 0.0304 | 0.1692 | | mse | 121588 | 219457 | 213677 | 4.68961e+06 | | total_time | 0.5216 | 0.6169 | 0.0051 | 0.0048 | ### Plot: ![](https://github.com/Nixtla/nixtla/blob/docs-figs-model-performance//action_files/models_performance/plots/plot_electricity-multiple-series_24_24_H_None_1.png?raw=true) ## Experiment 4: electricity-multiple-series ### Description: | variable | experiment | |:--------------|:-------------| | h | 168 | | season_length | 24 | | freq | H | | level | None | | n_windows | 1 | ### Results: | metric | timegpt-1 | timegpt-1-long-horizon | SeasonalNaive | Naive | |:-----------|------------:|-------------------------:|----------------:|---------------:| | mae | 465.532 | 346.984 | 398.956 | 1119.26 | | mape | 0.062 | 0.0437 | 0.0512 | 0.1583 | | mse | 835120 | 403787 | 656723 | 3.17316e+06 | | total_time | 0.597 | 1.0689 | 0.0057 | 0.0051 | ### Plot: ![](https://github.com/Nixtla/nixtla/blob/docs-figs-model-performance//action_files/models_performance/plots/plot_electricity-multiple-series_168_24_H_None_1.png?raw=true) ## Experiment 5: electricity-multiple-series ### Description: | variable | experiment | |:--------------|:-------------| | h | 336 | | season_length | 24 | | freq | H | | level | None | | n_windows | 1 | ### Results: | metric | timegpt-1 | timegpt-1-long-horizon | SeasonalNaive | Naive | |:-----------|--------------:|-------------------------:|----------------:|---------------:| | mae | 558.649 | 459.769 | 602.926 | 1340.95 | | mape | 0.0697 | 0.0566 | 0.0787 | 0.17 | | mse | 1.22721e+06 | 739135 | 1.61572e+06 | 6.04619e+06 | | total_time | 0.832 | 0.589 | 0.0058 | 0.0052 | ### Plot: ![](https://github.com/Nixtla/nixtla/blob/docs-figs-model-performance//action_files/models_performance/plots/plot_electricity-multiple-series_336_24_H_None_1.png?raw=true)