[MNT] maintenance & handover items for integration with sktime org

[fkiraly]

We are currently planning a maintenance handover of pytorch-forecasting to sktime. Both packages will remain separate, with pytorch-forecasting on the "models" level, and sktime providing framework integration.

This issue is to plan urgent maintenance items and handover (to sktime maintenance model).

Community input on todos and wishlist is also appreciated, e.g., what are "burning" items, suggested priorities.

Maintenance items

• [x] pyproject review, dependabot PRs
• [x] soft dependency review and potential isolation (e.g., optuna or matplotlib, not needed for NN models core)
  • https://github.com/jdb78/pytorch-forecasting/issues/1616
• [x] stale PR review and triage - highest priority
• [x] failing CI fixes
• [x] failing docs
• [x] python 3.11, 3.12 https://github.com/jdb78/pytorch-forecasting/issues/1301
• [x] EOL for python 3.8
• [x] windows compatibility https://github.com/jdb78/pytorch-forecasting/issues/1623
• [x] test that tutorial notebooks run, fix any breakage https://github.com/jdb78/pytorch-forecasting/pull/1621 https://github.com/jdb78/pytorch-forecasting/pull/1615
• [ ] optional: numpy 2 compatibility
• [ ] optional: resume conda releases, blocked since 0.10.3 https://github.com/jdb78/pytorch-forecasting/issues/1663

handover items

• [ ] issue triage
• [ ] stale PR review and triage - all
• [ ] operational handover
• [x] pointers to dev channels
• [ ] digital assets
• [ ] release pipeline

roadmap items for consideration or wishlist

• [ ] architectural review
• [ ] test coverage review (not just line coverage but case coverage)
• [ ] refactor model level tests to systematic test suite?
• [ ] consolidate torch based neural networks in pytorch-forecasting (e.g., move from sktime.networks)?
  • https://github.com/jdb78/pytorch-forecasting/issues/1618

[fkiraly]

FYI sktime developers who have worked on the integration, @benheid, @fnhirwa, @geetu040, @xinyuwuu, @yarnabrina

[fkiraly]

At the very start, we need to get the CI run again. Reason for failures:

• linting failures
• numpy 2 incompatibility at missing version bound, #1594
• optuna >= 3.3.0 incompatibility at missing version bound, #1595
• MPS backend memory failures https://github.com/jdb78/pytorch-forecasting/issues/1596

Suggested measures:

• allow force-push for maintainers, that allows PR that fix part of but not all failures - a single PR that fixes everything is a bad idea since there are too many independent failures.
• move the pyproject to pip from poetry, this makes work on CI easier https://github.com/jdb78/pytorch-forecasting/pull/1593
• lint with black, and change linting check to only look at changed files https://github.com/jdb78/pytorch-forecasting/pull/1597, https://github.com/jdb78/pytorch-forecasting/pull/1598
• add bounds to incompatible versions, fix later (also part of #1593)
• then fix the remaining failure, MPS backend issue

Once this is done, release 1.1.0.

Next, dependency management:

• support python 3.11, 3.12
• widen version bounds for numpy, then optuna, fix incompatibilities
• isolate dependencies that could be soft dependencies. Minimize core dep set.
• manage and refactor developer dep sets

[XinyuWuu]

Fixing tutorial notebooks is also very important: https://github.com/jdb78/pytorch-forecasting/pull/1599

[fkiraly]

of course - we won't know whether we fixed them though if the CI does not run

[XinyuWuu]

Regarding the dependency management, maybe we should not minimize core dep set.

Comparing to torch, lightning, optuna, other packages are too small to make a difference to installation time.

It could be a disaster if a lot a time has been spent to train the model, and then some soft dependency error comes out.

So maybe we should make sure that users have almost all the functionalities with the core dep set.

[fkiraly]

Agree, though even small dependencies pose a risk of reducing the degree of interoperability on the level of dependencies. Suppose you have small_package which has a very restrictive dependency set, or small_package2 which is the only dependency that never upgrades to numpy 2 or python 3.12.

Further, I wonder about how central optuna is here. It is important for tuning, but its purpose seems extraneous to the primary defining topic of deep learning based forecasters.

[XinyuWuu]

Further, I wonder about how central optuna is here. It is important for tuning, but its purpose seems extraneous to the primary defining topic of deep learning based forecasters.

It's only used in pytorch_forecasting/models/temporal_fusion_transformer/tuning.py to tune TemporalFusionTransformer. So I guess it's not so central.

[XinyuWuu]

Agree, though even small dependencies pose a risk of reducing the degree of interoperability on the level of dependencies. Suppose you have small_package which has a very restrictive dependency set, or small_package2 which is the only dependency that never upgrades to numpy 2 or python 3.12.

Yeah, that could be a problem. We could have two dep sets core and compatible and put those packages that don't fit in well in compatible set.