search

kubeflow / examples

A repository to host extended examples and tutorials
Apache License 2.0
1.41k stars 757 forks source link

Revamping the examples repository #1109

Open dpoulopoulos opened 6 months ago

dpoulopoulos commented 6 months ago

The kubeflow/examples repository is a valuable resource for demonstrating how to use Kubeflow for machine learning. However, it has fallen out of date, with examples using outdated libraries and lacking maintenance. I'd like to suggest a list of actions to address these issues and improve the overall quality and relevance of this repository:

  1. Restructure Repository Content
    • End-to-End Examples Only: Retain only the end-to-end examples in this repository.
    • Tool-Specific Examples: Move framework-specific examples to their corresponding repositories, where their respective working groups will be responsible for maintenance.
  2. Code Overhaul
    • Review and update existing examples: Update the code to use the latest versions of libraries and frameworks, ensuring compatibility with the current Kubeflow version.
    • Develop and integrate new examples: Identify and prioritize new examples that demonstrate cutting-edge use cases and advanced features of Kubeflow.
  3. Branching Model
    • Introduce Development Branch: Establish a develop branch (the default branch) for ongoing development work.
    • Release Management: Create specific release branches from the develop branch that follow Kubeflow releases.
  4. Best Practices and Repository Management (for example)
    • Main branch Protection: Protect the main repository to ensure stability.
    • PR Reviews and Merging: Enforce the practice of PR reviews before merging changes.
    • Templates for PRs and Issues: Introduce standardized templates for creating Pull Requests and Issues to streamline contributions.
  5. Documentation Updates
    • README: Update the README file to reflect the changes and provide clear instructions on how to use and contribute to the repository.
    • CONTRIBUTING: Revise the CONTRIBUTING file to include the new contribution guidelines and best practices.

If you agree with the proposal, I'd like to get involved. Based on this suggestion, let's start by identifying and categorizing existing examples into end-to-end and tool-specific. Then, we can create and configure a develop branch that will house the end-to-end examples and begin introducing the code and documentation changes we need.

StefanoFioravanzo commented 6 months ago

@dpoulopoulos Thanks for starting this issue. I couldn't agree more with all the action items you raised! Is there a particular sequenced priority that you would like to recommend?

cc some people who might be interested in this @andreyvelich @chasecadet @rimolive

kimwnasptd commented 6 months ago

Really nice proposal!

Regarding the structure, I don't have too much of a preference over e2e only ones. My only suggestion would be to think how we could make it easier for a new user coming in to know what a tutorial is about. For example, it's more helpful to organise them and say a tutorial is about hyper parameter tuning, rather than Katib.

Only because end users don't have the mapping of components in their mind, but tasks and flows they want to do and find similar examples to what they want to do