mediacloud / nyt-news-labeler

Tag news stories based on models trained on the NYT corpus.
Apache License 2.0
39 stars 12 forks source link

Disable mmap()'ping word2vec model (for now) #14

Closed pypt closed 4 years ago

pypt commented 4 years ago

Doesn't appear to work correctly when loaded in a Docker container: if multiple instances of an image get started (e.g. as a swarm service), neither containers seem to be able to load the word2vec model.

There are other ways to get around this, but for now it's not critical - we can just load the 4 GB module for each worker independently.

Ping @rahulbot.