Avoiding OOM errors

[jimkang]

Hi, I installed this via pip to try it out, and I'm getting OOM errors like this:

2022-09-08 16:35:25.958426: W tensorflow/core/framework/op_kernel.cc:1780] OP_REQUIRES failed at matmul_op_impl.h:728 : RESOURCE_EXHAUSTED: OOM when allocating tensor with shape[1,8,70151,70151] and type float on /job:localhost/replica:0/task:0/device:CPU:0 by allocator cpu

Is there a way I can tell spacy to tell tensorflow that it's OK to use more memory?

[MartinoMensio]

Hi @jimkang, When does this error happen? Are you using the vectors down in a pipeline using tensorflow?

I think you can have different types of issues:

• memory available actually finished:
  • you processed lots of documents with spaCy and did not set the 'enable_cache':False in the config options of universal_sentence_encoder. Therefore, a cache of document vectors is kept in TFHubWrapper. You can try to see if this resolves your problem when you initialise the spacy pipeline: nlp.add_pipe('universal_sentence_encoder', config={'enable_cache': False})
  • you may be trying to allocate a matrix that is too big (70151 x 70151) x 8. So please check and try with smaller batches
• memory still available, but tensorflow needs to be told directly to use more: if this is the case, try before importing and using spacy_universal_sentence_encoder to first configure tensorflow appropriately. However this seems unlikely as by looking online usually tensorflow has no limit and eats up everything that is available when processing on CPU

Best, Martino

[jimkang]

Thanks, Martino! It was that memory actually did run out.

After taking another look, I realized that the pipe expects a single sentence, rather than an entire document. (Also, strangely, when I call add_pipe I get 'universal_sentence_encoder' already exists in pipeline, so I used replace_pipe to set enable_cache.)

Thanks for confirming that it shouldn't normally run out of memory on a reasonably-sized sentence!