ServiceNow / Fast-LLM

Accelerating your LLM training to full speed

https://servicenow.github.io/Fast-LLM/

Other

37 stars 5 forks source link

Fix rope scaling when loading llama configs #36

Closed tscholak closed 1 week ago

tscholak commented 2 weeks ago

✨ Description

Certain Llama configs were crashing Fast-LLM. This fix seems to resolve the issue.

🔍 Type of change

Select all that apply:

[x] 🐛 Bug fix (non-breaking change that addresses a specific issue)

📝 Changes

Changed the value of a constant from False to None following a suggestion by @jlamypoirier

✅ Checklist

Make sure the following tasks are completed before submitting the PR:

General:

[x] 📜 I have read and followed the contributing guidelines.
[x] 🎉 The functionality is complete, and I have tested the changes.
[ ] 📝 I have updated the documentation if needed.
[ ] ⚠️ The change does not introduce any new issues (e.g., runtime warnings, type checker errors, linting problems, unhandled edge cases).
[ ] 🧩 I have commented my code, especially in hard-to-understand areas.

Dependencies and Configuration:

[ ] 🐋 I have updated the Docker configuration or dependencies, if applicable.
[ ] 🔄 I have ensured compatibility with the existing setup after dependency changes.

Testing:

[ ] 🧪 I have added or updated tests to cover my changes.
[ ] ✔️ New and existing tests pass locally with my changes.
[ ] 🚦 I have tested these changes on GPUs and verified training stability.
[ ] 🏋️ I have tested the changes on realistic training workloads, if applicable.

Performance Impact:

[ ] 📊 I have run benchmarks where applicable to evaluate the performance impact.
[ ] ✅ The benchmarks show no performance regression.
[ ] 🚀 The benchmarks indicate a potential performance improvement.
[ ] ⚠️ The benchmarks indicate a potential performance degradation.
[ ] 📈 I have provided benchmark results and detailed any performance impact below, if applicable.

📊 Performance Impact Details

N/A

📝 Additional Notes

N/A