updating architecture diagram

[vikas-saxena02]

Pull request for https://github.com/kubeflow/website/issues/3536

[vikas-saxena02]

@andreyvelich @thesuperzapper @juliusvonkohout some feedback would really be great!!

[andreyvelich]

/assign @kubeflow/kubeflow-steering-committee @kubeflow/wg-pipeline-leads @kubeflow/wg-notebooks-leads @kubeflow/wg-training-leads @kubeflow/wg-manifests-leads

[thesuperzapper]

Regarding "Kubeflow Universe" vs "Kubeflow Ecosystem", I'm not sure that "ecosystem" accounts for stuff that is not really part of Kubeflow but is commonly used alongside it (e.g. TensorFlow / PyTorch), because it kind of makes it sound like we own them. I kind of like the word "universe" as well but I'm open to change my mind.

---

We also might want to specify with brackets underneath each component what they actually do, similar to the less comprehensive introduction diagram that is already on the website:

[vikas-saxena02]

@andreyvelich I have just made the recommended changes except for Kubeflow Universe as I have same thoughts as @thesuperzapper on that one. @thesuperzapper added descriptions in brackets as you suggested.

[andreyvelich]

@thesuperzapper Yes, but ecosystem is mostly used term to explain that these ML projects have integrations with Kubeflow. For example, HuggingFace has a page which describe how to use Kubeflow Training Operator to run distributed training: https://huggingface.co/docs/transformers/main/en/perf_train_cpu_many#pytorchjob-specification-file cc @kubeflow/wg-training-leads

[thesuperzapper]

@andreyvelich perhaps we could use the term "Kubeflow Ladscape" similar to the "CNCF Landscape": https://landscape.cncf.io/

[andreyvelich]

Yes, maybe Landscape also could work. Then should we rename Kubeflow Components to Kubeflow Ecosystem ? Any other thoughts @kubeflow/kubeflow-steering-committee @vikas-saxena02 ?

[james-jwu]

I like "Kubeflow Ecosystem" at the top, and "Kubeflow Components" for the various KF projects. "Ecosystem" is used more than "Universe". See one example here for Spark: https://www.oreilly.com/library/view/building-data-streaming/9781787283985/81b1268e-23c2-492e-9710-2496c601d1de.xhtml

[vikas-saxena02]

@andreyvelich I would leave kubeflow components as it is as we use the same terminology in the structure of documentation of website. The top section is the only one where it's difficult for me to decide on my own.

[andreyvelich]

I like "Kubeflow Ecosystem" at the top, and "Kubeflow Components" for the various KF projects. "Ecosystem" is used more than "Universe". See one example here for Spark: https://www.oreilly.com/library/view/building-data-streaming/9781787283985/81b1268e-23c2-492e-9710-2496c601d1de.xhtml

I agree with James, ecosystem can be interpreted differently. It doesn't necessary mean that Kubeflow implements ML libraries (e.g. PyTorch and Tensorflow).

@vikas-saxena02 Maybe we can name middle box as Kubeflow Tools and in that section we can have: Kubeflow Components and External Add-Ons ?

[vikas-saxena02]

@andreyvelich good suggestion. I think we should rename the middle section to kubeflow tools and scaffolding in-line with current name.

[vikas-saxena02]

@james-jwu @andreyvelich @thesuperzapper, I was just wondering if you guys are happy for me to apply the following changes:

1. Rename top box to Kubeflow Ecosystem
2. Rename middle box to Kubeflow Tools and Scaffolding

[andreyvelich]

@james-jwu @andreyvelich @thesuperzapper, I was just wondering if you guys are happy for me to apply the following changes:

1. Rename top box to Kubeflow Ecosystem
2. Rename middle box to Kubeflow Tools and Scaffolding

That sounds good for to @vikas-saxena02 (Middle box - Kubeflow Scaffolding) . Any thoughts @james-jwu @kubeflow/kubeflow-steering-committee ?

[james-jwu]

Top box Kubeflow Ecosystem - SGTM

Middle box - "Scaffolding" is usually a framework that is designed to expose 3rd party code as services (e.g. Spring Boot), so it's probably not a good choice. "Tools" sounds a bit peripheral to me. "Kubeflow Components and External Add-Ons" ("External" can be dropped for brevity) sounds better to me.

[andreyvelich]

SGTM @james-jwu @vikas-saxena02

[vikas-saxena02]

@james-jwu @andreyvelich Changed Top Box to Kubeflow Ecosystem and MiddleBox to Tools as suggested

[vikas-saxena02]

I have the following thoughts:

• I would visually highlight Kubeflow Core Components compare to External Add-Ons and Integrations.
• remove Apache MXNet since this project was deprecated.
• change HoroVod -> Horovod
• Change HuggingFace -> Hugging Face
• Do we need explanation in stack diagram for every box (e.g. Certificate Manager) ? We can always explain it separately or link to the appropriate pages.

@andreyvelich

• for 1. Will making the External Add-Ons bio smaller (one column with the 4 boxes) help?
• 2,3 and 4 done
• The intention of adding description was to ensure that non-technical users (Managers, BAs etc) get some context to the related components. The diagram is a svg so we can even add links to the corresponding pages in the website

[andreyvelich]

for 1. Will making the External Add-Ons bio smaller (one column with the 4 boxes) help?

I was thinking that we should use another colour for Core Components. Also, should we just remove grey boxes from Core Components and External Add-ons and just name this box at the top ?

Also, we might want to avoid sharp corners like in prev diagram: https://www.kubeflow.org/docs/started/architecture/

[vikas-saxena02]

for 1. Will making the External Add-Ons bio smaller (one column with the 4 boxes) help?

I was thinking that we should use another colour for Core Components. Also, should we just remove grey boxes from Core Components and External Add-ons and just name this box at the top ?

Also, we might want to avoid sharp corners like in prev diagram: https://www.kubeflow.org/docs/started/architecture/

Cool.... I will make these changes today!!

[vikas-saxena02]

for 1. Will making the External Add-Ons bio smaller (one column with the 4 boxes) help?

I was thinking that we should use another colour for Core Components. Also, should we just remove grey boxes from Core Components and External Add-ons and just name this box at the top ?

Also, we might want to avoid sharp corners like in prev diagram: https://www.kubeflow.org/docs/started/architecture/

@andreyvelich done :)

[andreyvelich]

@vikas-saxena02 Please can you make font bigger for Core components and External Add-ons. I think, I was using 35 or 40. Also, should we change colour for boxes in integrations section so Core Kubeflow Components will be visually highlighted ?

[vikas-saxena02]

@vikas-saxena02 Please can you make font bigger for Core components and External Add-ons. I think, I was using 35 or 40. Also, should we change colour for boxes in integrations section so Core Kubeflow Components will be visually highlighted ?

Hi @andreyvelich do you want the font to be increased for just core components and external add-ons or everything?

[andreyvelich]

@vikas-saxena02 Please can you make font bigger for Core components and External Add-ons. I think, I was using 35 or 40. Also, should we change colour for boxes in integrations section so Core Kubeflow Components will be visually highlighted ?

Hi @andreyvelich do you want the font to be increased for just core components and external add-ons or everything?

Yes, only for core and external like in my example above: https://github.com/kubeflow/website/pull/3680#issuecomment-1979634805

[vikas-saxena02]

@andreyvelich sorry it took sometime to made the requested changes. I have ioncreased the font size for core components to highlight them as compared to others. Didnt made the font size for external add-ons same as core components because external add-ons are not maintained by us.

[vikas-saxena02]

Also, changed font sizes for everything to make them visible

[vikas-saxena02]

@andreyvelich @james-jwu @thesuperzapper @jbottum plz let me know if any more changes are ro be made.

[james-jwu]

@vikas-saxena02 In my previous comment, the suggestion is for the middle box to be "Kubeflow Components and External Add-Ons" ("External" can be dropped for brevity). Andrey lgtm'd in the next comment. I don't see this change reflected in the latest diagram.

[vikas-saxena02]

@vikas-saxena02 In my previous comment, the suggestion is for the middle box to be "Kubeflow Components and External Add-Ons" ("External" can be dropped for brevity). Andrey lgtm'd in the next comment. I don't see this change reflected in the latest diagram.

@james-jwu done :) Apologies, I somehow missed it.

[james-jwu]

@vikas-saxena02 In my previous comment, the suggestion is for the middle box to be "Kubeflow Components and External Add-Ons" ("External" can be dropped for brevity). Andrey lgtm'd in the next comment. I don't see this change reflected in the latest diagram.

@james-jwu done :) Apologies, I somehow missed it.

There is a misunderstanding here. What I meant was to rename the "Tool" in th left-middle box to "Kubeflow Components and External Add-Ons". Please see my linked comment on why I think "Tool" isn't a good choice.

[james-jwu]

/lgtm

[andreyvelich]

I recently made some changes to @vikas-saxena02 diagram to follow Kubeflow styling. What do you think?

[james-jwu]

I like the styling updates :)

[thesuperzapper]

@andreyvelich It might look better with a banner version of the Kubeflow logo (rather than square).

(the above image is an SVG)

[vikas-saxena02]

@andreyvelich just wondering if we should add Spark or Apache Spark or SparkOperator to the top left box as its now officially a part of kubeflow. Let me know your thoughts

[andreyvelich]

I applied suggested changes.

@andreyvelich just wondering if we should add Spark or Apache Spark or SparkOperator to the top left box as its now officially a part of kubeflow. Let me know your thoughts

@vikas-saxena02 We will add those components when Model Registry and Spark Operator will make the first release. cc @kubeflow/wg-data-leads

Please take a look again so we can merge it finally. /assign @kubeflow/kubeflow-steering-committee @kubeflow/wg-training-leads @kubeflow/wg-pipeline-leads @kubeflow/wg-notebooks-leads @kubeflow/wg-deployment-leads @zijianjoy

[tenzen-y]

Thank you! /lgtm

[andreyvelich]

I squashed all of the commits and add @vikas-saxena02 as co-author. Please review it once again.

[james-jwu]

/lgtm

[andreyvelich]

Thanks everyone for your help working on this! Let's merge our first version. /approve

[google-oss-prow[bot]]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: andreyvelich, vikas-saxena02

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[content/en/docs/started/OWNERS](https://github.com/kubeflow/website/blob/master/content/en/docs/started/OWNERS)~~ [andreyvelich] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment

[vikas-saxena02]

I am just wondering if the team who manages the LinkedIn profile of Kubeflow Org can do a post on the updated architecture.

[james-jwu]

@vikas-saxena02 You can reach out to Amber.