search

pytorch / torchtune

PyTorch native finetuning library
https://pytorch.org/torchtune/main/
BSD 3-Clause "New" or "Revised" License
4.35k stars 440 forks source link

Update Qwen2.5 configs #1999

Closed joecummings closed 1 week ago

joecummings commented 1 week ago
  1. I turned activation checkpointing off for all 0.5B models and for 1.5B LoRA models. No point.
  2. I turned on memory logging

Everything else is cosmetic.

pytorch-bot[bot] commented 1 week ago

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchtune/1999

Note: Links to docs will display an error until the docs builds have been completed.

:white_check_mark: No Failures

As of commit 8596e5acce71642a37ef81eab34216a37fc590ed with merge base 18d97f0eb15f50b0d2e569c1ba136de8907edabb (image): :green_heart: Looks good so far! There are no failures yet. :green_heart:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

joecummings commented 1 week ago

I thought we use underscores instead of periods?: #1863 (comment)

Yeah this is a misleading comment. We do use underscores for model builders, but the model should just get downloaded to a directory with the exact same name as the model on the Hub.