Open fanqiNO1 opened 2 weeks ago
If the LLM is too big to be loaded in a single GPU, we need device_map = 'auto' to avoid OOM.
device_map = 'auto'
According to the issue #715.
If the LLM is too big to be loaded in a single GPU, we need
device_map = 'auto'to avoid OOM.According to the issue #715.