Pull request for moverscore_v2 GPU modification

AIPHES / emnlp19-moverscore

MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distance

MIT License

192 stars 31 forks source link

Pull request for moverscore_v2 GPU modification #26

Closed jabirshabbir closed 1 year ago

forrestbao commented 1 year ago

This PR will allow moverscore_v2.py to run with GPU support.

andyweizhao commented 1 year ago

Thank you so much!

forrestbao commented 1 year ago

We noticed something very interesting that even after this PR, the GPU utility rate is still very low. One CPU core is occupied most of the time while the GPU is almost idle. We think the reason is on the implementation of tokenization, padding, etc.

https://github.com/AIPHES/emnlp19-moverscore/blob/0459a3b3b3bd73baa0cc515a355228ee5a2887e0/moverscore_v2.py#L78-L86

andyweizhao commented 1 year ago

Inefficient use of GPU is because the current WMD algorithm computes one instance at a time, even though inputs are given in the batch format. So GPU remains idle until WMD finishes a batch of computation instance by instance.