search

thoth-station / thoth-application

Thoth-Station ArgoCD Applications
GNU General Public License v3.0
12 stars 22 forks source link

Possibly revisit how we host AICoE Python packages #1013

Closed fridex closed 3 years ago

fridex commented 3 years ago

Is your feature request related to a problem? Please describe.

Currently, we run an HTTP server that serves a static content from within the directory. To help to scale and maintaining multiple python packages, we could have a look for an alternative. The HTTP server needs to be rebuilt each time a package is about to be added which does not integrate very well.

Describe the solution you'd like

Deploy one or multiple pypiserver instances to serve python packages. As pypiserver can serve python packages, we would gain possiblity to upload built artifacts directly from AICoE-CI. A use case for this could be integration of https://github.com/thoth-station/thoth-application/issues/955 when we would build numpy and publish it on our index using AICoE-CI pipeline.

See relevant issue on pypiserver: https://github.com/pypiserver/pypiserver/issues/382

A downside of this solution is authentication exposure to the world.

Describe alternatives you've considered

Keep serving Python packages using the index stated.

Additional context

We could also start being package agnostic. The current url to our AICoE index is specific to "tensorflow". This can be misleading if we build or host our own python artifacts.

https://tensorflow.pypi.thoth-station.ninja/

fridex commented 3 years ago

Worth to have a look at https://pypi.org/project/private-pypi/

fridex commented 3 years ago

Another candidate: https://www.devpi.net/

goern commented 3 years ago

/livecycle active

fridex commented 3 years ago

See https://github.com/thoth-station/httpd-aicoe-container/issues/28#issue-649136008 for checking correctness of the index.

fridex commented 3 years ago

Closing, Pulp deployment is WIP which will be used as a package index: https://github.com/operate-first/support/issues/176

/close

sesheta commented 3 years ago

@fridex: Closing this issue.

In response to [this](https://github.com/thoth-station/thoth-application/issues/1013#issuecomment-853879426): >Closing, Pulp deployment is WIP which will be used as a package index: https://github.com/operate-first/support/issues/176 > >/close Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.