search

thoth-station / adviser

The recommendation engine for Python software stacks and Dependency Monkey in project Thoth.
https://thoth-station.github.io
GNU General Public License v3.0
34 stars 13 forks source link

Create a tool that can assist in the pipeline unit placement #1076

Closed fridex closed 3 years ago

fridex commented 4 years ago

Is your feature request related to a problem? Please describe.

As adviser would grow, it might become hard to maintain the relative order of pipeline units - boots, sieves, steps, strides and wraps in the respective modules. It might be a good idea to introduce a tool that can optimize pipeline units placement in the __init__.py files. This optimization would be run once offline (not during an adviser run) and would determine the relative order in which pipeline units should be stated in the __init.py files.

An example could be:

https://github.com/thoth-station/adviser/blob/95e44f54e30e5a75ab081dc58c30a7764c435e5b/thoth/adviser/boots/__init__.py#L34-L51

Should UbiBoot be placed before VersionClashBoot? What would be the performance gain/drain?

Describe the solution you'd like

Each pipeline unit could add additional configuration (e.g. overall computation cost, how many queries and how expensive queries it performs). This configuration could be used by the described tool that could act as a scheduling optimizer to optimize what pipeline units should be stated sooner. The problem looks like reducible to traveling salesman problem.

Additional context

See the respective documentation section that states an algorithm that could be coded to optimize adviser pipeline unit placement.

https://github.com/thoth-station/adviser/pull/1075/files

sesheta commented 3 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

/lifecycle stale

fridex commented 3 years ago

This is superseded by prescriptions that provide a declarative way to maintain dependencies across pipeline units:

https://thoth-station.ninja/docs/developers/adviser/prescription.html#should-include-dependencies

/close

sesheta commented 3 years ago

@fridex: Closing this issue.

In response to [this](https://github.com/thoth-station/adviser/issues/1076#issuecomment-822323033): >This is superseded by prescriptions that provide a declarative way to maintain dependencies across pipeline units: > >https://thoth-station.ninja/docs/developers/adviser/prescription.html#should-include-dependencies > >/close Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.