search

intel / xFasterTransformer

Apache License 2.0
355 stars 61 forks source link

Does xFasterTransformer supports Falcon model? #74

Closed amir1m closed 9 months ago

amir1m commented 10 months ago

Hello, Planning to run PEFT tuned Falcon model on a CPU (Intel(R) Xeon(R) Platinum 8167M CPU @ 2.00GHz, dockerized Linux). However, the README doesn't mention Falcon, are only listed models supported?

Thanks.

jackywei commented 10 months ago

Yes, right now, we don't have the official support for Falcon, but it is in our plan now. BTW, if you have time or interest, I think you can also consider adding Falcon support based on xFT, since the Falcon has similar network with other supported ones, and we have good API, welcome to add more model support into xFT :). you will get best perf. cheers.

pujiang2018 commented 10 months ago

Falson was planned, but since we haven't got the strong requirement, thus delayed the support. And BTW: 8167 is too old, I don't think it will have a good performance without AMX.

amir1m commented 10 months ago

Falson was planned, but since we haven't got the strong requirement, thus delayed the support. And BTW: 8167 is too old, I don't think it will have a good performance without AMX.

Hi @pujiang2018 , Which CPU model would you recommend instead?

Thanks.

jackywei commented 10 months ago

better to use 94xx/84xx/64xx Xeon Processor: 84xx/64xx(4th Generation Intel® Xeon® Scalable Processors) - https://ark.intel.com/content/www/us/en/ark/products/series/228622/4th-generation-intel-xeon-scalable-processors.html 94xx(Intel® Xeon® CPU Max Series) - https://ark.intel.com/content/www/us/en/ark/products/series/232643/intel-xeon-cpu-max-series.html

Duyi-Wang commented 9 months ago

You can join our WeChat group to stay informed about our latest model support initiatives. Your assistance in adjusting the model support arrangements is also greatly welcomed.