Training: Add Fine-Tune API Docs

[andreyvelich]

Related: https://github.com/kubeflow/training-operator/issues/2013 This is draft PR for our new Fine-Tune API in Kubeflow Training Operator. We will work on the page structure in this Google doc to finalise it: https://docs.google.com/document/d/18PuuaDRISj5mlrBn1GJrxwuB6Z5zTtXKpVbLUIeLx-8/edit?usp=sharing.

[andreyvelich]

I added content from the Google doc and one tutorial. Please let me know what do you think. /assign @StefanoFioravanzo @kubeflow/wg-training-leads @deepanker13 @kuizhiqing

[andreyvelich]

/hold for review

[google-oss-prow[bot]]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: andreyvelich

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[OWNERS](https://github.com/kubeflow/website/blob/master/OWNERS)~~ [andreyvelich] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment

[andreyvelich]

I addressed your comments @StefanoFioravanzo. Regarding this comment:

Also, what is the expected output?

How we can show the expected output ? Our LLM trainer doesn't support any output yet: https://github.com/kubeflow/training-operator/blob/master/sdk/python/kubeflow/trainer/hf_llm_training.py#L178, so we need to work in the future to understand how user should consume the fine-tuned model. E.g. exporting to S3 or to other storage. cc @johnugeorge @deepanker13

[StefanoFioravanzo]

so we need to work in the future to understand how user should consume the fine-tuned model.

Issue + KF 1.10 tag? :)

[andreyvelich]

@StefanoFioravanzo I believe, I addressed all of your comments. Does it look good to you ? /assign @johnugeorge @deepanker13 @tenzen-y

[StefanoFioravanzo]

@andreyvelich yes it does thank you!

[deepanker13]

@andreyvelich the links in fine-tuning.md are giving 404 page not found. Am I missing something?

[andreyvelich]

@andreyvelich the links in fine-tuning.md are giving 404 page not found. Am I missing something?

@deepanker13 Did you check these links via Website preview: https://deploy-preview-3718--competent-brattain-de2d6d.netlify.app/ ?

[deepanker13]

@andreyvelich the links in fine-tuning.md are giving 404 page not found. Am I missing something?

@deepanker13 Did you check these links via Website preview: https://deploy-preview-3718--competent-brattain-de2d6d.netlify.app/ ?

@andreyvelich it's working with the preview. Thanks for the awesome documentation! /lgtm

[StefanoFioravanzo]

@andreyvelich shall we merge this one?

[andreyvelich]

Sure, let's merge it. Thanks everyone for review! /hold cancel