bhoov / exbert

A Visual Analysis Tool to Explore Learned Representations in Transformers Models
http://exbert.net
Apache License 2.0
584 stars 51 forks source link

while i am doing the data processing? #9

Open zz860808 opened 4 years ago

zz860808 commented 4 years ago

Is there any example about the format of the txt file? Because i always got an error in the file "create_faiss.py".

Creating Embedding faiss files in /home/ubuntu/will/exbert/server/data/processing/embeddings from /home/ubuntu/will/exbert/server/data/processing/embeddings/embeddings.hdf5
Traceback (most recent call last):
  File "create_faiss.py", line 145, in <module>
    main(args.directory)
  File "create_faiss.py", line 114, in main
    embedding_ce = CorpusEmbeddings(str(embedding_hdf5))
  File "/home/ubuntu/will/exbert/server/data/processing/corpus_embeddings.py", line 304, in __init__
    self.embedding_dim = self.embeddings['0001'].shape[-1]
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "/home/ubuntu/anaconda3/envs/exbert/lib/python3.6/site-packages/h5py/_hl/group.py", line 264, in __getitem__
    oid = h5o.open(self.id, self._e(name), lapl=self._lapl)
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py/h5o.pyx", line 190, in h5py.h5o.open
KeyError: "Unable to open object (object '0001' doesn't exist)"  

I have no idea about why the key error happened... Or someone can tell me how can i solve the problem. thanks very much.