search

huggingface / Google-Cloud-Containers

Hugging Face Deep Learning Containers (DLCs) for Google Cloud
https://hf.co/docs/google-cloud
Apache License 2.0
128 stars 16 forks source link

PEFT Finetune Container #93

Open NiteeshReddyB opened 1 month ago

NiteeshReddyB commented 1 month ago

How to pass a custom jsonL for finetuning job like by just uploading the jsonl file to GCS or upload the Jsonl file on huggingface and also in the args how to pass evaluation args

alvarobartt commented 1 month ago

Hi here @NiteeshReddyB, sorry I didn't quite get it, could you please add more details?

Do you mean reading a JSONL dataset to be used for fine-tuning? If that's the case, it would depend on which library are you using e.g. TRL's CLI at the moment only supports datasets being loaded from the Hugging Face Hub which is what we showcase in this example.

Alternatively, for the kwargs the same, depends on the library you use for training within the Hugging Face DLC; e.g. for TRL CLI you can provide a YAML file with the fine-tuning configuration as per TRL's documentation.

Anyway, if you could provide more details that would be great to be able to help you further! Thanks in advance 🤗

NiteeshReddyB commented 1 month ago

Yes, for a JSONL dataset to be used for fine-tuning. Got it, thanks.

alvarobartt commented 1 month ago

FYI I've created a dedicated example for this issue for you to check; but long-story-short you can use a custom fine-tuning script based on TRL and modify the loading function for the datasets, allowing you to load the JSONL file from a GCS bucket via load_dataset("json", data_files=[...]).

https://gist.github.com/alvarobartt/b2a3067d69622a0ef05aee3a113ce73b

Hope that helps! 🤗

[!NOTE] The example provided above is a minimal and fast reproduction of the issue to showcase that it can be done, while that may not be the best way, since we're reusing the dataset_name argument only; but ideally one should create a custom script suited to its needs. I think that being able to easily run custom code on top of the containers is really useful, so I'll be creating a new dedicated example in the repository!

alvarobartt commented 1 month ago

P.S. More information on the from_local_script method on python-aiplatform.

NiteeshReddyB commented 1 month ago

Thank you for sharing the links; they were very helpful. Does Hugging Face offer prebuilt containers for CI/CD on any cloud platforms like Azure or AWS that can be used to deploy or fine-tune models? Additionally, are there prebuilt containers available for deploying multi-models from Hugging Face?

alvarobartt commented 1 month ago

Anytime! 🤗

Does Hugging Face offer prebuilt containers for CI/CD on any cloud platforms like Azure or AWS that can be used to deploy or fine-tune models?

Yes, any of the Hugging Face DLCs can be used for any workload, including CI/CD; here you can find the useful links for each Cloud Provider with available DLCs:

More information at https://github.com/aws/deep-learning-containers, https://aws.amazon.com/ai/hugging-face/, and https://huggingface.co/docs/sagemaker/en/index

More information at https://huggingface.co/docs/google-cloud/index, and https://cloud.google.com/deep-learning-containers/docs/choosing-container#hugging-face

Alternatively, we also have integrations with other providers as Azure.

Additionally, are there prebuilt containers available for deploying multi-models from Hugging Face?

With multi-models do you mean multi-modal? If that's the case you can do so with the TGI DLCs e.g. the Hugging Face DLC for TGI on Google Cloud us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-text-generation-inference-cu121.2-2.ubuntu2204.py310; as those come with VLM support for certain architectures, e.g. IDEFICS.

NiteeshReddyB commented 1 month ago

Thanks for sharing that really helps. Any useful links for Azure please share you have any.