search

ServiceNow / embedding-propagation

Codebase for Embedding Propagation: Smoother Manifold for Few-Shot Classification. This is a ServiceNow Research project that was started at Element AI.
Apache License 2.0
208 stars 21 forks source link

problem about multi gpu #23

Closed invokerpicker closed 2 years ago

invokerpicker commented 2 years ago

Hi, thank you for your release! The adjacency matrix computing needs all embeddings together. However the code is using dataparallel on multi gpus. In this situation, the adjacency matix computing can only uses part of the embeddings in a single gpu. Using single gpu and multi gpus has different results. I'm wondering the solution for such a problem.

prlz77 commented 2 years ago

Hi @invokerpicker, thanks for the interest in our work! We perform embedding propagation after extracting embeddings with DataParallel such that all of them are together in the same GPU.

  1. Extract embeddings: https://github.com/ElementAI/embedding-propagation/blob/master/src/models/finetuning.py#L140
  2. Embedding prop: https://github.com/ElementAI/embedding-propagation/blob/master/src/models/finetuning.py#L145

I hope this solves your question.

invokerpicker commented 2 years ago

Thanks!