piskvorky / gensim

Topic Modelling for Humans
https://radimrehurek.com/gensim
GNU Lesser General Public License v2.1
15.63k stars 4.37k forks source link

PerplexityMetric example from docs doesn't print to terminal in my conda environment #2936

Open dwrodri opened 4 years ago

dwrodri commented 4 years ago

Problem description

What are you trying to achieve?

I am trying to print the perplexity of an LDA model to stdout/stderr in a shell.

What is the expected result?

Some floating point number printed to stdout/stderr

What are you seeing instead?

I see no output in either stderr or stdout

Steps/code/corpus to reproduce

Include full tracebacks, logs and datasets if necessary. Please keep the examples minimal ("minimal reproducible example").

$ cat test.py
from gensim.models.callbacks import PerplexityMetric
from gensim.models.ldamodel import LdaModel
from gensim.test.utils import common_corpus, common_dictionary

# Log the perplexity score at the end of each epoch.
perplexity_logger = PerplexityMetric(corpus=common_corpus, logger='shell')
lda = LdaModel(common_corpus, id2word=common_dictionary, num_topics=5, callbacks=[perplexity_logger])
$ python test.py
$

Versions

Please provide the output of:

import platform; print(platform.platform())
import sys; print("Python", sys.version)
import struct; print("Bits", 8 * struct.calcsize("P"))
import numpy; print("NumPy", numpy.__version__)
import scipy; print("SciPy", scipy.__version__)
import gensim; print("gensim", gensim.__version__)
from gensim.models import word2vec;print("FAST_VERSION", word2vec.FAST_VERSION)
Python 3.6.11 | packaged by conda-forge | (default, Aug  5 2020, 20:09:42)
[GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import platform; print(platform.platform())
Linux-5.4.0-7642-generic-x86_64-with-debian-bullseye-sid
>>> import sys; print("Python", sys.version)
Python 3.6.11 | packaged by conda-forge | (default, Aug  5 2020, 20:09:42)
[GCC 7.5.0]
>>> import struct; print("Bits", 8 * struct.calcsize("P"))
Bits 64
>>> import numpy; print("NumPy", numpy.__version__)
NumPy 1.19.1
>>> import scipy; print("SciPy", scipy.__version__)
SciPy 1.5.2
>>> import gensim; print("gensim", gensim.__version__)
gensim 3.8.3
>>> from gensim.models import word2vec;print("FAST_VERSION", word2vec.FAST_VERSION)
FAST_VERSION 0

I have also dumped my conda env to a pastebin, here is the link.


I feel like I'm missing something obvious here, but as far as I can tell this example should be printing perplexity in my terminal, but it isn't. Many thanks in advance to the people who take the time to reply to this issue.

dwrodri commented 4 years ago

Update: I've managed to determine that the code is following the correct path all the way to the function call from the logging library. This reinforces my concern that I'm just not setting something up properly with Python's logging library. I'll return to this later with more details.