search

thoth-station / adviser

The recommendation engine for Python software stacks and Dependency Monkey in project Thoth.
https://thoth-station.github.io
GNU General Public License v3.0
34 stars 13 forks source link

Recommend the image to be used with a software stack without providing runtime environment #1943

Open pacospace opened 3 years ago

pacospace commented 3 years ago

Is your feature request related to a problem? Please describe. As Data Scientist,

I want to build an AI pipeline where all my steps have different software stacks and runtime environments. Currently I have to choose runtime environment, but I don't know what is best for my software stack (best might be performance or latest for example).

The question I have is:

Can I receive the recommendation on the runtime to be used for a certain software stack (beside the software stack), so that I know the base image to use for building an image to use in the steps of my AI Pipeline?

Describe the solution you'd like Have a recommendation when a runtime environment is not in input.

Describe alternatives you've considered For steps that do not require performance in particular, just use a runtime environment I have. Submit different requests to thoth with same software stack, different runtime environments known to thoth, with performance recomendation type, and select the one that gives the highest score?

Additional context Related-To: https://github.com/thoth-station/jupyterlab-requirements/issues/229

fridex commented 3 years ago

Is your feature request related to a problem? Please describe. As Data Scientist,

I want to build an AI pipeline where all my steps have different software stacks and runtime environments. Currently I have to choose runtime environment, but I don't know what is best for my software stack (best might be performance or latest for example).

The question I have is:

Can I receive the recommendation on the runtime to be used for a certain software stack (beside the software stack), so that I know the base image to use for building an image to use in the steps of my AI Pipeline?

We can prioritize https://github.com/thoth-station/adviser/issues/1926 for this. In that case, we could have a look at the requirements a Thoth user uses and we will try to match them with container image analysis of Thoth s2i images. In that case, we will just check which s2i/jupyter notebook/... image uses a superset of Python packages that the user wants and we can recommend the base image for them (also considering other things like python version a user is running).

Describe alternatives you've considered For steps that do not require performance in particular, just use a runtime environment I have. Submit different requests to thoth with same software stack, different runtime environments known to thoth, with performance recomendation type, and select the one that gives the highest score?

This is something the system could do on its own - https://github.com/thoth-station/adviser/issues/1533 - basically, if a request has no runtime environment specified, it can trigger multiple advisers with different runtime environments and see which runtime environment wins.

goern commented 3 years ago

@pacospace what is the next step here?

pacospace commented 3 years ago

@pacospace what is the next step here?

That we take the recommendation from adviser and automatically populate the thoth.yaml so that AICOE CI uses the correct base image when building images for AI Pipelines: https://github.com/thoth-station/jupyterlab-requirements/issues/345

sesheta commented 3 years ago

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

/lifecycle rotten

fridex commented 3 years ago

/remove-lifecycle rotten /triage accepted

sesheta commented 3 years ago

Rotten issues close after 30d of inactivity. Reopen the issue with /reopen. Mark the issue as fresh with /remove-lifecycle rotten.

/close

sesheta commented 3 years ago

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

/lifecycle rotten

sesheta commented 3 years ago

@sesheta: Closing this issue.

In response to [this](https://github.com/thoth-station/adviser/issues/1943#issuecomment-926553689): >Rotten issues close after 30d of inactivity. >Reopen the issue with `/reopen`. >Mark the issue as fresh with `/remove-lifecycle rotten`. > >/close Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
codificat commented 1 year ago

/sig stack-guidance