Fix Llama3 PEFT Merge dtype

aws-samples / amazon-bedrock-samples

This repository contains examples for customers to get started using the Amazon Bedrock Service. This contains examples for all available foundational models

https://aws.amazon.com/bedrock/

MIT No Attribution

509 stars 265 forks source link

Fix Llama3 PEFT Merge dtype #219

Closed jpil-builder closed 3 months ago

jpil-builder commented 3 months ago

Issue #, if available:

Description of changes: When Merging the model adapters for llama 3 using any other torch dtype besides float16 is giving NAN values in the model.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.