search

fixie-ai / thefastest.ai

Website with current metrics on the fastest AI models.
MIT License
30 stars 3 forks source link

have you tried tenstorrent card based inference? #11

Closed arjunkrishna closed 1 month ago

arjunkrishna commented 4 months ago

I am trying to figure out if tenstorrent cards also provide the same throughput for fraction of the groq accerator cards. I do not see many comparisons.

groq accerator card.. it seems they are 20k each https://www.mouser.com/ProductDetail/BittWare/RS-GQ-GC1-0109?qs=ST9lo4GX8V2eGrFMeVQmFw%3D%3D

tenstorrent grayskull cards.. it seems like they are around $700 each. https://tenstorrent.com/cards/

if you have access to such setup then adding tenstorrent based testing would be good. then home users can try out such a card for home use as well.

juberti commented 4 months ago

FLOPs seem similar, grayskull seems to have more memory. This isn't something we could take on as there's probably a fair amount of work to get a model to work well on the card, but I'm guessing that we might see someone stand up a service using these cards.

juberti commented 1 month ago

Closing, will revisit if we see a provider with this hardware.