search

ludwig-ai / ludwig

Low-code framework for building custom LLMs, neural networks, and other AI models
http://ludwig.ai
Apache License 2.0
11.19k stars 1.19k forks source link

Add custom `prepare_for_training` logic to ECD model for LLM encoder adapter initialization #3874

Closed jeffkinnison closed 10 months ago

jeffkinnison commented 10 months ago

The LLM model type initializes the adapter weights and quantization at training time using LLM.prepare_for_training. When LLMEncoder was added, ECD model did not have a corresponding prepare_for_training method, so adapter initialization occurred at encoder initialization. This PR adds ECD.prepare_for_training, which brings LLMEncoder adapter initialization to parity with LLM models.

github-actions[bot] commented 10 months ago

Unit Test Results

  6 files  ±0    6 suites  ±0   14m 13s :stopwatch: -6s 12 tests ±0    9 :heavy_check_mark: ±0    3 :zzz: ±0  0 :x: ±0  60 runs  ±0  42 :heavy_check_mark: ±0  18 :zzz: ±0  0 :x: ±0 

Results for commit 768aaaf7. ± Comparison against base commit 89a032f6.

:recycle: This comment has been updated with latest results.