search

sidhantls / adaptive-rank-selection-svd

Implementation of Adaptive Rank Selections for Low-Rank Approximation of Language Models
2 stars 1 forks source link

multi-gpu for llama-2-13b #2

Closed sidhantls closed 2 weeks ago

sidhantls commented 1 month ago

currently, cannot run on single gpu for llama-2-13b. leads to OOM after replacing low-rank layers. implement on multi-gpu