search

kubeflow / training-operator

Distributed ML Training and Fine-Tuning on Kubernetes
https://www.kubeflow.org/docs/components/training
Apache License 2.0
1.62k stars 700 forks source link

KEP-2170: Create model and dataset initializers #2303

Closed andreyvelich closed 3 weeks ago

andreyvelich commented 4 weeks ago

Fixes: https://github.com/kubeflow/training-operator/issues/2210

I created model and dataset initializers. Initially, we will only support HF for the demo purposes. I will create dedicated issue to support more providers.

/assign @kubeflow/wg-training-leads @varshaprasad96 @akshaychitneni @deepanker13 @helenxie-bit @Electronic-Waste @saileshd1402 @kannon92

coveralls commented 4 weeks ago

Pull Request Test Coverage Report for Build 11517758882

Details

Totals Coverage Status
Change from base Build 11507477280: 0.0%
Covered Lines: 77
Relevant Lines: 77
💛 - Coveralls
kannon92 commented 4 weeks ago

Should we consider unit or e2e tests for this?

andreyvelich commented 4 weeks ago

Should we consider unit or e2e tests for this?

Yeah, I will open dedicated issue for it.

andreyvelich commented 4 weeks ago

I wonder if we can add some comment somewhere to let users know we only support downloading models/datasets from HuggingFace now?

Yes, we are planning to add supported dataset and model providers to the website. Additionally, I will create tracking issue to support more providers (S3, GCS, etc.)

andreyvelich commented 3 weeks ago

Are there any other comments before we can move forward with this initial PR ? /assign @kannon92 @Electronic-Waste @deepanker13 @varshaprasad96 @saileshd1402

google-oss-prow[bot] commented 3 weeks ago

@andreyvelich: GitHub didn't allow me to assign the following users: varshaprasad96, saileshd1402.

Note that only kubeflow members with read permissions, repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time. For more information please see the contributor guide

In response to [this](https://github.com/kubeflow/training-operator/pull/2303#issuecomment-2438957151): >Are there any other comments before we can move forward with this initial PR ? >/assign @kannon92 @Electronic-Waste @deepanker13 @varshaprasad96 @saileshd1402 Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
google-oss-prow[bot] commented 3 weeks ago

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: tenzen-y

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[OWNERS](https://github.com/kubeflow/training-operator/blob/master/OWNERS)~~ [tenzen-y] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment