search

bmabey / pyLDAvis

Python library for interactive topic model visualization. Port of the R LDAvis package.
BSD 3-Clause "New" or "Revised" License
1.8k stars 361 forks source link

(v3.4.1) IndexError: index 46308 is out of bounds for axis 1 with size 17797 #253

Open ooker777 opened 1 year ago

ooker777 commented 1 year ago

I get 

IndexError: index 46308 is out of bounds for axis 1 with size 17797

I'm using pyLDAvis v3.4.1 on Python 3.10.10 on Windows. Here is the full code:

import pickle
from gensim import models
import pyLDAvis.gensim

id2word = pickle.load(open('../data/id2word.pkl','rb'))
cv_vecs = pickle.load(open('../data/cv_vecs.pkl','rb'))
cv_corpus = pickle.load(open('../data/cv_corpus.pkl','rb'))
lda =  models.LdaModel.load('../data/gensim_lda.model')

viz = pyLDAvis.gensim.prepare(topic_model=lda, corpus=cv_corpus, dictionary=id2word, mds='mmds')

You can check the input files here

msusol commented 4 months ago

Correcting the code for v 3.4.1: pyLDAvis.gensim -> pyLDAvis.gensim_models

import pickle
from gensim import models
#import pyLDAvis.gensim
import pyLDAvis.gensim_models as gensimvis

id2word = pickle.load(open('../data/id2word.pkl','rb'))
cv_vecs = pickle.load(open('../data/cv_vecs.pkl','rb'))
cv_corpus = pickle.load(open('../data/cv_corpus.pkl','rb'))
lda =  models.LdaModel.load('../data/gensim_lda.model')

#viz = pyLDAvis.gensim.prepare(topic_model=lda, corpus=cv_corpus, dictionary=id2word, mds='mmds')
viz = gensimvis.prepare(topic_model=lda, corpus=cv_corpus, dictionary=id2word, mds='mmds')

Do you still get the same error?

msusol commented 4 months ago

FYI /data/cv_vecs.pkl is not present in zip file (if needed)

I get an error trying to evaluate using your model


Traceback (most recent call last):
  File "/Users/marksusol/DataScience/GitHub/pyLDAvis/issue253data/issue253.py", line 9, in <module>
    lda =  models.LdaModel.load('./data/gensim_lda.model')
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/marksusol/DataScience/GitHub/pyLDAvis/venv/lib/python3.11/site-packages/gensim/models/ldamodel.py", line 1663, in load
    result = super(LdaModel, cls).load(fname, *args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/marksusol/DataScience/GitHub/pyLDAvis/venv/lib/python3.11/site-packages/gensim/utils.py", line 487, in load
    obj._load_specials(fname, mmap, compress, subname)
  File "/Users/marksusol/DataScience/GitHub/pyLDAvis/venv/lib/python3.11/site-packages/gensim/utils.py", line 529, in _load_specials
    val = np.load(subname(fname, attrib), mmap_mode=mmap)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/marksusol/DataScience/GitHub/pyLDAvis/venv/lib/python3.11/site-packages/numpy/lib/npyio.py", line 427, in load
    fid = stack.enter_context(open(os_fspath(file), "rb"))
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: './data/gensim_lda.model.expElogbeta.npy'
``
msusol commented 4 months ago

Can't reproduce, consider stale.