search

stanfordnlp / GloVe

Software in C and data files for the popular GloVe model for distributed word representations, a.k.a. word vectors or embeddings
Apache License 2.0
6.86k stars 1.51k forks source link

Saving checkpoint on SIGUSR1 signal #61

Closed jbojar closed 3 years ago

jbojar commented 7 years ago

This modification allows to request saving a checkpoint at the end of current iteration by sending SIGUSR1 signal to glove process: kill -s SIGUSR1 <PID>.

Implementation starts a dedicated signal handling thread, which waits for SIGUSR1 signal using sigwait(...) function.

Kornel commented 7 years ago

+1, that's a nice solution when you start training with a large amount of iterations and can't afford storing multiple checkpoints. Using a signal you can checkpoint whenever you wish

AngledLuffa commented 4 years ago

Interesting concept. Is this likely to be portable? I have next to 0 experience programming C for Windows, but there's some talk of making GlOvE run on Windows.