Closed jbottum closed 1 month ago
I am also in favor of at least trying to write proper English, so no nonsense abbreviations such as apps instead of applications.
Regarding the terminology we could separate between applications and services (istio, knative cert-manager etc.)
Thanks Josh!
Once we have all the changes to the diagram, the person who originally made this for us is ready to update.
I really think we need to make categories and terminology match what we share. Otherwise, people will wonder what is correct the presentation/share or the website.
Julius, yes!
We shouldn’t be using abbreviations in documentation or on the website.
Additional, acronyms should be used after the first mention and in parentheses. We should not assume everyone knows what they mean. And there could be multiple meanings for the same acronym.
Examples: Machine Learning (ML), Cloud Native Computing Foundation (CNCF)
This way we make sure we put the full name at first mention in a section and put the associated acronym with it.
On Fri, Jun 30, 2023 at 5:53 AM Julius von Kohout @.***> wrote:
I am also in favor of at least trying to write proper English, so no nonsense abbreviations such as apps instead of applications.
Regarding the terminology we could separate between applications and services (istio, knative cert-manager etc.)
— Reply to this email directly, view it on GitHub https://github.com/kubeflow/website/issues/3536#issuecomment-1614485151, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABPJ3ZL3TOPZCH3BDEU335DXN2V3LANCNFSM6AAAAAAZY5CZRI . You are receiving this because you were mentioned.Message ID: @.***>
-- Sent from Gmail Mobile
Hi team, Can I get the original source for the image (e.g. draw.io fille or the raw file for tool which it was made on) and I am more than happy to modify it as per the recommendations. Personally believe KFServing needs to changed to Kserve as well. Otherwise I am happy to recreate the same. PS: Btw, new contributor and user of kubeflow here and happy to assist with documentation while I learn about the same.
@vikas-saxena02 and others, the Kubeflow website uses Docsy which supports a number of ways to generate diagrams dynamically, the main options we should consider are:
This makes it much easier to update the diagram in the future as it will be defined at code/markdown. Also, they produce "interactive" diagrams, whose text can be selected (good for accessibility and SEO), and they can include links (for example, to the docs for each component of Kubeflow).
Feel free to experiment with this by cloning the kubeflow/website
repo, and following the local development guide to get a local version running on your laptop. (NOTE: you might need to update the config.toml
to enable the diagram plugins)
Also, it can be a bit hard to find the markdown file for the "architecture page", for reference, it's here:
I’ve got the original. I can send it over. ~Amber
On Thu, Sep 14, 2023 at 4:23 PM Vikas Saxena @.***> wrote:
Hi team, Can I get the original source for the image (e.g. draw.io fille or the raw file for tool which it was made on) and I am more than happy to modify it as per the recommendations. Personally believe KFServing needs to changed to Kserve as well. Otherwise I am happy to recreate the same. PS: Btw, new contributor and user of kubeflow here and happy to assist with documentation while I learn about the same.
— Reply to this email directly, view it on GitHub https://github.com/kubeflow/website/issues/3536#issuecomment-1720170722, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABPJ3ZPH6WYFII7CDNBKDBTX2NYTZANCNFSM6AAAAAAZY5CZRI . You are receiving this because you were mentioned.Message ID: @.***>
@akgraner thanks !! that would be great. I do not have access to kubeflow slack channel as of yet. So if you can attach it here or send it through email, it would be great!! my email Id is vikas.saxena.2006@gmail.com
Another question I had was how can I get access to kubeflow's slack channel?
@thesuperzapper thanks for shortlisting mermaid and markdown, that makes life easier if I were to draw it from scratch. Docsy supports a bunch of graphic libraries for diagram so wasn't sure which one to go with.
@jbottum @thesuperzapper @akgraner can someone assign the issue to me so that there are no duplicate efforts on same. I am unable to do it myself
/assign @vikas-saxena02
Don't we want to use https://www.drawio.com/ to create our diagrams ?
This is an open-source tool and I believe it is possible to import existing png
file that was created using drawio
to make changes.
As I can see, docsy also has integrations with drawio
: https://www.docsy.dev/docs/adding-content/diagrams-and-formulae/#diagrams-with-diagramsnet
@andreyvelich the problem with draw.io
is that it creates a PNG, so is not able to have clickable links inside it, like I was saying in https://github.com/kubeflow/website/issues/3536#issuecomment-1720196054
I kind of agree with @thesuperzapper. draw.io comes with its own overhead of versioning controlling the draw.io file. if you chose to store file as .svg file, most of the times, it fails to open image in an editable way. I will continue with Mermaid for now and see if its easier to use, otherwise we can always switch back to draw.io
Just letting everyone know that I am currently familiarising myself with docsy and Mermaid and will start work on this in a day or two.
@thesuperzapper @vikas-saxena02 Are we planing to add any clickable links inside our diagrams ?
@andreyvelich not as of now but this can be future functionality. Happy to reconsider if majority agrees to use draw.io. For me its the following reasons to prefer mermaid over draw.io
Quick question here, is Fairing still maintained or has been discontinued like Chainer. The last update on fairing was in August 2021
@juliusvonkohout @thesuperzapper @akgraner @andreyvelich , I have got few other question
@vikas-saxena02 Regarding your first question, yes, istio
is a hard dependency of Kubeflow, so it does not make sense for it to be under "add-ons"
.
In fact, I have a PR to move it to the "multi-tenancy"
section, which we should merge ASAP:
@juliusvonkohout @thesuperzapper @akgraner @andreyvelich , I have got few other question
- Is istio really an add-on as there seems to be no option for installing kubeflow without it?
- Has anyone got a conceptual view of how the new architecture diagram should look like? If not, I will probably raise a work in progress PR shortly for your review. Happy to work it out ovet google meet based on your timing
@vikas-saxena02 We can check diagram from the recent "What is Kubeflow?" presentation: https://docs.google.com/presentation/d/16fGvHzBLGtRwN6qKrxkFBDsDxfBXchdedAUshC3NINs/edit#slide=id.g25617a67d82_1_7.
This represents the main components to run ML with Kubeflow in an easy way.
We can also have additional Stack diagram which shows full list of tools that Kubeflow uses (e.g. Istio, Cert Manager, Knative)
Thanks @thesuperzapper I am assuming PR will be approved and istio will be classified as a component. I will raise a work in progress PR in a day or two.
btw do you have some insights if Kserve will follow the same route and be classified as a component rather than an add-on
cc: @akgraner @jbottum @juliusvonkohout @andreyvelich
@vikas-saxena02 at this stage, KServe is a separate project (all be it, also controlled by the Linux Foundation).
So for now, there's no reason to give it specific preference in our architecture diagram, or even really include it unless we are also including other alternatives for serving.
@thesuperzapper I am actually including the serving tools described here https://www.kubeflow.org/docs/external-add-ons/serving/ in a separate box for External Add-Ons The only exception is TensorFlow Bathch Prediction which is no longer supported as of version 0.6. Happy to avoid adding all the external add-ons if that is the advise
or even really include it unless we are also including other alternatives for serving.
@thesuperzapper I would disagree with that.
Although, KServe is a separate project for now, Serving Working Group is still part of Kubeflow. KServe community invest time to make sure that KServe can run with Kubeflow manifests. Also, KServe/KFServing was part of Kubeflow since 2019 and many users are still considering KServe as part of Kubeflow. Thus, to not confuse them, I would include KServe in Kubeflow diagram until we find community agreement that we should remove it.
Additionally, we work with Serving Working Group to discuss our future around this.
Without Serving component on architecture diagram, Kubeflow has a huge gap for MLOps lifecycle.
What do you think @johnugeorge @jbottum @james-jwu @zijianjoy @yuzisun @ckadner
cc @kubeflow/wg-training-leads @kubeflow/wg-notebooks-leads @kubeflow/wg-pipeline-leads
Hi guys, I could not get mermaid working so switched to draw.io instead. Any feedback on the
new proposed architecture diagram
could comments, you might add seldon into add-ons, pipelines could have two components - argo and tekton. it might make sense to make pipelines and katib like notebooks and training operators. then perhaps central dashboard, istio and dex could be grouped together (with some other common items i.e. web apps).
Thanks @jbottum I will make the chnages later today. Btw, do you think we should mention kubeflow-kale as an add-on as well?
@vikas-saxena02 Should we use same colour as here: https://www.kubeflow.org/docs/started/architecture/ ? I believe it uses same colours as Kubeflow logo. Also, we can reduce empty spaces in your stack diagram and make text bigger, so users can easily read it.
I don't think we should include Kale, since we already removed this add-on: https://github.com/kubeflow/website/pull/3554
So we are going to have 2 diagrams:
Is that correct ?
I think we should keep it clean, and as close to this one https://www.kubeflow.org/docs/started/architecture/ as possible. It matches the others on the site. If you give me the final one that has all the updates I’ll get it updated (which was my original ask.)
Please let me know when all the updates are added and all the changes are represented, then I’ll have that original one updated to reflect all the changes.
Thanks!
On Thu, Sep 28, 2023 at 5:28 AM Andrey Velichkevich < @.***> wrote:
@vikas-saxena02 https://github.com/vikas-saxena02 Should we use same colour as here: https://www.kubeflow.org/docs/started/architecture/ ? I believe it uses same colours as Kubeflow logo. Also, we can reduce empty spaces in your stack diagram and make text bigger, so users can easily read it.
I don't think we should include Kale, since we already removed this add-on: #3554 https://github.com/kubeflow/website/pull/3554
So we are going to have 2 diagrams:
- This one for high-level architecture: #3536 (comment) https://github.com/kubeflow/website/issues/3536#issuecomment-1737888654
- This one for Kubeflow stack: #3536 (comment) https://github.com/kubeflow/website/issues/3536#issuecomment-1738212732
Is that correct ?
— Reply to this email directly, view it on GitHub https://github.com/kubeflow/website/issues/3536#issuecomment-1738884414, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABPJ3ZNEQRA6G3DHIIHNVSTX4VGLDANCNFSM6AAAAAAZY5CZRI . You are receiving this because you were mentioned.Message ID: @.***>
@vikas-saxena02 i would change "on-prem" to the proper term "on-premises". As far as i know fairing is dead.
@juliusvonkohout @akgraner I will endeavour to complete the architecture diagram by the weekend and will update the recent one here as well as in a "work in progress PR" Apologies about disappearing for a week as I was down with flu
- update architecture page on kubeflow.org #3536 (comment)
@andreyvelich I guess we will only have the kubeflow stack one in the offical documentation . The other one is just for presentation.
Also, thanks for highlighting the color theme as I was also thinking on similar lines,
@vikas-saxena02 i would change "on-prem" to the proper term "on-premises". As far as i know fairing is dead.
Hi @juliusvonkohout please let me know if Fairing needs to be removed from the stack and i will action it. In other words, is there an offical confirmation for Fairing to be dead, then its better to remove it from the stack
@akgraner do you have some insights if fairing is dead
Also, I am removing spartacus as reporting is not being maintained anymore
@akgraner @andreyvelich @jbottum @thesuperzapper @juliusvonkohout can someone please advise me the hex colour code for the blue colour on Kubeflow website. I am working on aligning the colours to that. Here is an updated draft of the image I will be making the Addons section a bit smaller. But any review here is appreciated.
I am thinking of raising a WIP (work in progress PR), am I allowed to change the OWNERS file and put you guys as approvers/reviewers?
@jbottum @akgraner @thesuperzapper @andreyvelich @juliusvonkohout here is the final version of the image
Happy to remove fairing if required
Please remove fairing. Also change Training Operators
to Training operator
Few questions.
Feature store
? Please remove fairing. Also change
Training Operators
toTraining operator
Will Do this evening (I am in Australia) and send the updated diagram.
Btw, Training Operators
is actually the term on kubeflow official website https://www.kubeflow.org/docs/components/training/
Few questions.
- I am not aware of Elyra. Is it active?
I took this from the documentation on kubeflow.org. Here is the link https://www.kubeflow.org/docs/external-add-ons/elyra/ I was of the understanding that this is the GUI to create pipelines in kubeflow
- Are we referring to any specific
Feature store
?
Again this one has been taken from official documentation of kubeflow available at https://www.kubeflow.org/docs/external-add-ons/feature-store/ . Currently the documentation s referencing Feast but I left it as a general item in architecture as there may be plans to support more of such products.
Happy to remove both if that is the general consensus.
Removed fairing.
I don't think, we should indicate all component-related info to this architecture (e.g. TFJob, PyTorchJob for Training Operators, Goptuna for Katib, VSCode for Kubeflow Notebooks). This is overwhelming, and doesn't give much value for the user, who just want to understand which components Kubeflow has. This can be explained in individual components architectures in the appropriate website sections.
This diagram should just provide name of the Kubeflow Apps (Notebooks, Training Operators, Katib, KFP, PVC Viewer, PodDefaults, Tensorboard, Profile Controller).
@andreyvelich I have a different opinion, from a user perspective its better to highlight what all sub-components or functionalities are there to the original application/component. For new users who are in process of selecting the right ML tool, it will provide a better understanding of the kubeflow ecosystem.
@andreyvelich I have a different opinion, from a user perspective its better to highlight what all sub-components or functionalities are there to the original application/component. For new users who are in process of selecting the right ML tool, it will provide a better understanding of the kubeflow ecosystem.
Please can you join today's community call to discuss this ?
@andreyvelich I saw your message now. I am in Sydney so your message came at 11 pm for me. Hence I missed the community call. I am happy to jump on a google meet if that solves the problem. Can you please advise your availability? cc: @jbottum
@andreyvelich can we discuss this in the next community call. I will try my best to attend it
@vikas-saxena02 It is on Tuesday 8 am PST
Sorry guys, its at 2 am my time so I often miss it.
Apologies for being late to post this but the PR has this updated diagram since January. Please let me know any feedback
Hi all, I had to cancel my PR as i couldnt do a rebase to signoff all previous commits. So created a new branch and did the PR all over agin. Here is the link to the new PR https://github.com/kubeflow/website/pull/3680
Closing because we merged:
/close
@thesuperzapper: Closing this issue.
The current Kubeflow architecture diagram on Kubeflow.org is out of date: https://www.kubeflow.org/docs/started/architecture/
Proposal:
6.Kubeflow applications and scaffolding need an extensive re-write to match apps and common names in manifest github repo, https://github.com/kubeflow/manifests
DO WE WANT THE NAMES IN THE REPO TO MATCH THE WEBSITE EXACTLY? IF NOT, WHAT DO WE EXCLUDE ON THE WEBSITE. (see list from repo below...which use the term COMPONENT RATHER THAN APPS.
see 2nd list from repo below for Common, which also uses the term COMPONENT
Other a. We might want to consider a way to minimize the info and updates on kubeflow.org b. We might want to drive users to the github manifest repo for current updates. We could then focus on updating info on the github repo.
Related - Julius has provided this diagram on Kubeflow, which is a good technical diagram but not a replacement for the existing diagram on kubefllow.org. still, we figure out if/how we can use it (see page 8). https://kccnceu2023.sched.com/event/1HyY8
@akgraner @juliusvonkohout @kimwnasptd @annajung