Closed SnoozingSimian closed 1 year ago
This document https://github.com/NVIDIA/FasterTransformer/blob/main/docs/gptneox_guide.md demonstrates how to build FT source codes and run the gemm test.
Hi, I got the answer from bert- guide docs, because I have same issue on BERT models.
"If you want to use the library in other directory, please generate this file according to your setting and copy it to your working directory."
When I move the gemm_config.in file to working directory, where I start the triton server, the message was not appeard.
Thanks @minsuuuuuuuub I also reached the same answer.
So in order to use the gemm_config.in
we need to place it in the directory where the triton server is being run from, it just does a local lookup.
Weird how this is not mentioned anywhere in the GPT docs.
Discussed in https://github.com/triton-inference-server/fastertransformer_backend/discussions/48