ludwig-ai / ludwig

Low-code framework for building custom LLMs, neural networks, and other AI models
http://ludwig.ai
Apache License 2.0
10.97k stars 1.18k forks source link

Support for freezing pretrained vision model layers with regex #3981

Closed ethanreidel closed 1 month ago

ethanreidel commented 3 months ago

Allows the user to input a regular expression in the yaml config which freezes specific layers of a pretrained model. Adds new CLI option "pretrained_summary" to let users access string representations of model layers for freezing via regex. Currently all pretrained torchvision models are accessible.

trainer: layers_to_freeze_regex: (regex here)

ludwig pretrained_summary -m (model name here)

(I am aware that the collect_summary CLI command is similar, however it only accepts a preexisting directory so I thought creating a separate command to strictly output layer names was appropriate for this feature.)

Closes #3733

Future plans -> expand this capability to implement gradual unfreezing

Test: pytest tests/ludwig/modules/test_regex_freezing.py

github-actions[bot] commented 3 months ago

Unit Test Results

  6 files  ±0    6 suites  ±0   18m 36s :stopwatch: + 4m 15s 12 tests ±0    7 :heavy_check_mark:  -   2    5 :zzz: +  2  0 :x: ±0  60 runs  ±0  30 :heavy_check_mark:  - 12  30 :zzz: +12  0 :x: ±0 

Results for commit 4e62b925. ± Comparison against base commit 4b07ce4e.

This pull request skips 2 tests. ``` tests.regression_tests.benchmark.test_model_performance ‑ test_performance[ames_housing.gbm.yaml] tests.regression_tests.benchmark.test_model_performance ‑ test_performance[mercedes_benz_greener.gbm.yaml] ```

:recycle: This comment has been updated with latest results.

ethanreidel commented 2 months ago

@saad-palapa

arnavgarg1 commented 1 month ago

I would also recommend installing pre-commit via pip install pre-commit, then running pre-commit install within the Ludwig repo. That will help fix some of the pre-commit related styling errors here: https://results.pre-commit.ci/run/github/163346054/1714080170.FEtyFVFyR8m3t6xqARcnDQ

arnavgarg1 commented 1 month ago

@ethanreidel Let us know when this is ready for re-review!

ethanreidel commented 1 month ago

Hi @arnavgarg1! Thanks for checking in. The PR is good for re-review. 👍