The transformer-based tasks will inevitably take time to run, but some of them could probably be made more efficient. For example, I have the label entities task set up for Cuda, but I'm not 100% sure it is actually using Cuda properly. @tcrick had an idea for how to refactor this task to ensure it is taking full advantage of the GPU. Try that change and then see if we can improve processing time for the others as well.
Whisper Transcript from that part of the conversation between John and Tyler below.
But, um, because like one thing I'm definitely thinking is that it's looking like the model is predicting entities in one sentence at a time, right?
Yes. Yeah.
So that's where we're losing the CUDA benefit.
Because I think that if I think that should be able to be done in parallel and that's where you're making use of all the CUDA cores because it's massively parallel, right?
So that might be, it might be losing the entire GPU sort of benefit by doing that in a series instead of parallel.
I'm not totally positive about that, but...
SPEAKER 2 0:54:18
That could explain a lot.
You could also explain why this function, I just have the impression, I haven't tested it, I have the impression that it's running slower than some of my previous iterations on this function.
And that could definitely be why.
The other thing too is I know like a lot of, like you have to be pretty careful with how you handle like iteration with Pandas stuff.
...
SPEAKER 1 0:55:20
Cool. Um, that is like a hundred percent a thing I love to do.
So, okay, cool.
SPEAKER 2 0:55:31
Um, I'd like to be plugged into that too, because like I'm trying to learn how to do this kind of thing better.
So yeah, like anything that you figure out that will make this more efficient, like I want to know what it is you're doing.
The transformer-based tasks will inevitably take time to run, but some of them could probably be made more efficient. For example, I have the label entities task set up for Cuda, but I'm not 100% sure it is actually using Cuda properly. @tcrick had an idea for how to refactor this task to ensure it is taking full advantage of the GPU. Try that change and then see if we can improve processing time for the others as well.
Whisper Transcript from that part of the conversation between John and Tyler below.