Goldritter
commented
3 years ago Short update: I tried to make the function "get-word-embeddings-for-token-information" more in touch with Python and remove the nth call but I get the same behavior described above.
(defn get-word-embeddings-for-token-information [token-information ^String path]
(py. (t/squeeze (t/stack
(py/with [r (t/no_grad)]
(py/get-item
((memonized-get-bert-model path)
(:indexed-tokenz-tensor token-information)
(t/tensor [(:segments-tensors token-information)]))
2))
:dim 0) :dim 1) permute 1 0 2))
jjtolton
cnuernber
Hi,
I try to using Pytorch und Hugginfaces Transformers to create Bert Word-Embeddings. Here is the code of the namespace which creates the Embbedings:
If I now use the call
(BERT/get-second-last-layer (BERT/token-embeddings-of-sentences "Die 87-jährige Franziska R. wird wegen zunehmender Dyspnoe und Hustenanfällen aus der Augenklinik in die internistische Abteilung eines Klinikums verlegt." BERT/german-bert-path))in a namespace wehere I used (:require [python.bert.test :as BERT]) it works fine without any problems.But if I want to work with a sequence of sentences with
(map #(BERT/get-second-last-layer (BERT/token-embeddings-of-sentences %1 BERT/german-bert-path)) [<Many sentences>]))or(map #(BERT/token-embeddings-of-sentences %1 BERT/german-bert-path) [<Many sentences>]))the program stuck randomly and I need to restart the entire REPL. On average after 3 calls will the program stuck.I get no Error Message and if I stop the execution and use the call again the program stuck again. The same happens if I used a loop or a for.
I tried many methods to transform the Tensor directly to a list and work with it or I work only with the Tensor or I save the loaded models directly as variable in the namespace instead to use Memonize, but I always encounter the same weird behavior.
Has anybody an idea what I#m doing wrong?
Best Goldritter