Open kevinintel opened 1 month ago
I am interested in this as well. keep me in the loop on the tasks that need to be done. Right now I am trying to get llama 3.1 405B 8bit running on Gaudi-TGI, and llama 3.1 405B GGUF with speculative decoding using llama 3.1 8B working on Xeon
I am interested in this as well. keep me in the loop on the tasks that need to be done. Right now I am trying to get llama 3.1 405B 8bit running on Gaudi-TGI, and llama 3.1 405B GGUF with speculative decoding using llama 3.1 8B working on Xeon