search

kermitt2 / delft

a Deep Learning Framework for Text https://delft.readthedocs.io/
Apache License 2.0
388 stars 64 forks source link

Find a way to disable the ELMo/BERT caching mechanism in "production" mode #66

Closed oterrier closed 4 years ago

oterrier commented 4 years ago

The contextual embeddings (ELMo or BERT) caching mechanism using a lmdb database is really nice especially in training mode because it saves a lot of time after the 1st epoch. Anyway when you want to use intensively your trained model in "production" making a lot of prediction over a long period of time, the lmdb database can potentially grows infinitively when providing massively unseen texts to the Tagger.tag() method It could be usefull to have a way of disabling the cache, like for example with a use_cache boolean flag that could overwrite the default behaviour (not during training but in production)

Not sure that it is crystal clear ... Let me know

kermitt2 commented 4 years ago

Thank you @oterrier, this is very clear and actually using the lmdb cache for ELMo and BERT embeddings was only intended for training. I think for prediction it never makes no sense to use it (it can very quickly consume too much disk space) and should always been deactivated. A use_cache boolean is indeed necessary !