search

weaviate / t2v-transformers-models

This is the repo for the container that holds the models for the text2vec-transformers module
BSD 3-Clause "New" or "Revised" License
40 stars 27 forks source link

Adding objects to weaviate with custom model fails #4

Closed StefanBogdan closed 3 years ago

StefanBogdan commented 3 years ago

Adding objects to weaviate with custom model fails because nltk.download('punkt') is NOT executed for custom models. This happens because nltk.download('punkt') is only in the download.py script, which is not executed for custom models.

>>> try:
..      client.data_object.create(data_obj, "Article")
..  except weaviate.UnexpectedStatusCodeException as error:
..      print(error.json['error'][0]['message'])

fail with status 500: 
**********************************************************************
  Resource punkt not found.
  Please use the NLTK Downloader to obtain the resource:

  >>> import nltk
  >>> nltk.download('punkt')

  For more information see: https://www.nltk.org/data.html

  Attempted to load tokenizers/punkt/PY3/english.pickle

  Searched in:
    - '/root/nltk_data'
    - '/usr/local/nltk_data'
    - '/usr/local/share/nltk_data'
    - '/usr/local/lib/nltk_data'
    - '/usr/share/nltk_data'
    - '/usr/local/share/nltk_data'
    - '/usr/lib/nltk_data'
    - '/usr/local/lib/nltk_data'
    - ''
**********************************************************************