search

tingofurro / summac

Codebase, data and models for the SummaC paper in TACL
https://arxiv.org/abs/2111.09525
Apache License 2.0
85 stars 24 forks source link

Performance #9

Closed UntotaufUrlaub closed 1 year ago

UntotaufUrlaub commented 1 year ago

Hi,

I am wondering, whether this metric runs as time efficient as possible at my machine. I assume running on the gpu is faster than on cpu. What would be the best indicator to see if my machine is fully utilized? nvidia-smi only reports about 1 GB of additional gpu memory usage during evaluation, but close to 100% GPU-Util. Is this indicating max gpu usage? I am curious because of the low memory usage, even though I processed a batch of documents and summaries. That took a few minutes to finish. Are there any parameters to tweak performance, like batch sizes etc.?

kind regards

tingofurro commented 1 year ago

Hey @UntotaufUrlaub,

Thanks for reaching out. A 100% GPU utilization is a good sign. If you are using SummaCZS, it should use the batched computation (with build_images), you could then play around with the batch size to see if you can fit more samples at a time into your GPU.

If you use SummaC-Conv, I believe the default is not yet batched, because it involved some refactoring. If you end up batching the main SummaC-Conv function and see performance gains, feel free to add it as a pull request!

UntotaufUrlaub commented 1 year ago

Hi @tingofurro,

thank you!

When you say playing around, do you mean tweaking the code? I did not see any setting, like a constructor parameter of SummaCZS, which could be adjusted. Or is this hidden in the **kwargs?