The speed issue with the GTATask.

arcee-ai / mergekit

Tools for merging pretrained large language models.

GNU Lesser General Public License v3.0

4.88k stars 446 forks source link

The speed issue with the GTATask. #388

Open daidaiershidi opened 3 months ago

daidaiershidi commented 3 months ago

Thank you for open-sourcing such a great tool. When executing the GTATask, I use LoadTensor twice to load additional tensors, but this makes the execution very slow. How can I solve this issue?

shuang0112 commented 3 months ago

I have the same problem, have you solved?

daidaiershidi commented 3 months ago

I have the same problem, have you solved?

no. >_<

cg123 commented 3 months ago

Could you please give some more details on what exactly you're doing? It's hard to say what's going on without knowing more. In general IO is actually the most time consuming part of the merging process though. If you're doing 3X the amount of tensor loads I wouldn't be too shocked if it got a lot slower. Maybe there's something to do to mitigate that though.