GPU Recommendation

[RickArcher108]

With Tigros's help I managed to get Whisperer working and I've been batch processing hundreds of recordings. I have a 12GB AMD GPU. With this, Whisperer can process 2 recordings at once, using the large model, which requires 4gb for each process. So if I set it to do two at once, it uses 8gb with 4 to spare. But doing this, I find the GPU fans whir loudly and my PC occasionally crashed and restarts. So I've been batch processing just one at a time.

Someone who supports my project has offered to buy me a move powerful GPU. I'd prefer NVidia because an NVidia came with my Dell and my computer consultant had trouble installing the AMD. But still, there are so many choices. Can anyone recommend a model costing less than $2K with at least 24gb of RAM which might be best for my purposes?

My PC case is a mini-tower, so 3-fan GPUs probably won't fit into it.

Thanks.

[tigros]

the only Nvidia card with 24GB is RTX 4090, that's top of the line today, and about $2K.

But for much less a used 3080 would be best value, that's only 10 or 12 GB tho.

on second thought those won't fit in your case, cases don't cost much, but lots of work for your consultant.

correction an RTX 3090 Ti also has 24GB, maybe find a used one.

[RickArcher108]

Hmmm. That happens to be 13.26" / 336.80 mm in length, I have a mini-tower, and most mini tower cases have a maximum GPU length of 280mm to 330mm. Any idea what the most powerful GPU I could fit into a mini-tower might be?

From: tigros @.> Sent: Wednesday, September 27, 2023 7:13 PM To: Const-me/Whisper @.> Cc: Rick Archer @.>; Author @.> Subject: Re: [Const-me/Whisper] GPU Recommendation (Issue #179)

the only Nvidia card with 24GB is RTX 4090, that's top of the line today, and about $2K.

But for much less a used 3080 would be best value, that's only 10 or 12 GB tho.

— Reply to this email directly, view it on GitHubhttps://github.com/Const-me/Whisper/issues/179#issuecomment-1738267343, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A7JTFILVU7BPMAIDIYJLD3LX4S6KNANCNFSM6AAAAAA5J263DU. You are receiving this because you authored the thread.Message ID: @.**@.>>

[tigros]

Hi Rick,

Yes I realized that after replying, fyi you see at the bottom a link "view it on GitHub' you should get into the habit of clicking that and replying there.

a quick search i found this: MSI - NVIDIA GeForce RTX 4090 SUPRIM LIQUID

no idea if the radiator will fit somewhere tho, otherwise maybe a new case.

[emcodem]

I'd definitely get a bigger case with a few fans when running automated processing that utilize 100% GPU constantly. On the other hand i'd probably not process such stuff at home because the power costs might be higher than renting some gpu in the cloud. IF i'd like to run many whisper transcriptions at home, i'd invest the money into some Apple M1 and use the whispercpp project instead of const-me version, it would save somewhere between 2 and 400W power consumption and probably deliver more throughput at the same time.

[Const-me]

I don’t know an answer to your question, but here’s some comments, hopefully relevant.

The DLL compiled from my source code has an API to share model’s tensors across multiple threads, to do concurrent transcribes. This way you don’t need many gigabytes of VRAM, but none of the usage examples demonstrate how to use that. The two key pieces are eGpuModelFlags.Cloneable flag for loading a model, and iModel.Clone method.

On paper, the best GPU under $2K is Radeon RX 7900 XTX. The best nVidia GPUs are 4090 and 3090 Ti, the performance in my library is equal. 3090, 3080 Ti, and 12 GB 3080 are very close. See these tables, I think the most important number is “Memory bandwidth GB/s”: NV 4000, NV 3000, AMD 7000.

[RickArcher108]

Thanks. After much research, I decided to get a new computer with a 4090 in it. It was time for a new computer anyway.