search

dice-group / WHALE

0 stars 0 forks source link

Noctua_2_WHALE_Memory_Mapping_RDFa_10M-7844215 job stopped working (float issue) #3

Closed sshivam95 closed 2 weeks ago

sshivam95 commented 2 weeks ago

RDFa dataset is giving memory overload issue when ran with dice-embeddings.

sshivam95 commented 2 weeks ago

Fixed by implementing memory map using memmappickle.mmapdict

sshivam95 commented 2 weeks ago

Test on the extracted files --> Running on extracted 10M Noctua 1 - 3066765

sshivam95 commented 2 weeks ago

Done