Open LiuChen19960902 opened 1 year ago
If you change the model architecture, you should modify the FT source codes first.
If you change the model architecture, you should modify the FT source codes first.
Can you please provide me with some references? I would greatly appreciate it.
You need to check what files do you need to modify, like https://github.com/NVIDIA/FasterTransformer/blob/main/src/fastertransformer/models/bert/Bert.cc and other related files.
You need to check what files do you need to modify, like https://github.com/NVIDIA/FasterTransformer/blob/main/src/fastertransformer/models/bert/Bert.cc and other related files.
Thank you very much for your advice, I will go and study it.
你的问题解决了吗? 我也碰到了该问题
你的问题解决了吗? 我也碰到了该问题
emm,没有,在研究源码中....
感觉应该有适配的,不然实际场景绝大多数模型都是不可用的呀
感觉应该有适配的,不然实际场景绝大多数模型都是不可用的呀
目前我还没找到,毕竟现在用LLM的话,感觉基本上也不会自定义结构。
demo给的好像只可以输出output_hidden_state这个特征向量。 如果是用bert做简单的二分类,这个你知道怎么做么
demo给的好像只可以输出output_hidden_state这个特征向量。 如果是用bert做简单的二分类,这个你知道怎么做么
暂时还不知道
How should I use FasterTransformer Triton to deploy my custom model, such as adding other structures after BERT? Assuming my model structure is defined like this:
At this point, I have some questions: