Closed jimkang closed 1 year ago
Hi @jimkang, When does this error happen? Are you using the vectors down in a pipeline using tensorflow?
I think you can have different types of issues:
'enable_cache':False
in the config options of universal_sentence_encoder
. Therefore, a cache of document vectors is kept in TFHubWrapper
. You can try to see if this resolves your problem when you initialise the spacy pipeline: nlp.add_pipe('universal_sentence_encoder', config={'enable_cache': False})
spacy_universal_sentence_encoder
to first configure tensorflow appropriately. However this seems unlikely as by looking online usually tensorflow has no limit and eats up everything that is available when processing on CPUBest, Martino
Thanks, Martino! It was that memory actually did run out.
After taking another look, I realized that the pipe expects a single sentence, rather than an entire document. (Also, strangely, when I call add_pipe
I get 'universal_sentence_encoder' already exists in pipeline
, so I used replace_pipe
to set enable_cache
.)
Thanks for confirming that it shouldn't normally run out of memory on a reasonably-sized sentence!
Hi, I installed this via pip to try it out, and I'm getting OOM errors like this:
Is there a way I can tell spacy to tell tensorflow that it's OK to use more memory?