Trouble with the packaging tutorial

[Shaktigupta931]

OS version

Ubunu 22.04

Python version

Python 3.9.12

Pip version

pip 21.2.4

Guide link

https://packaging.python.org/tutorials/packaging-projects

Problem description

I have some other files in my directory but it is not laoding when I am importing the wheel file of that sms_model | | init.py | casa_wallet_flask_sms_api_svc.py | requirements.txt | svc_classifier.pkl |___ word2vec.model

Error message

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/shakti_gupta/anaconda3/lib/python3.9/site-packages/svc_model/casa_wallet_flask_sms_api_svc.py", line 8, in <module>
    model = Word2Vec.load("word2vec.model")
  File "/home/shakti_gupta/anaconda3/lib/python3.9/site-packages/gensim/models/word2vec.py", line 1953, in load
    model = super(Word2Vec, cls).load(*args, **kwargs)
  File "/home/shakti_gupta/anaconda3/lib/python3.9/site-packages/gensim/utils.py", line 486, in load
    obj = unpickle(fname)
  File "/home/shakti_gupta/anaconda3/lib/python3.9/site-packages/gensim/utils.py", line 1460, in unpickle
    with open(fname, 'rb') as f:
  File "/home/shakti_gupta/anaconda3/lib/python3.9/site-packages/smart_open/smart_open_lib.py", line 188, in open
    fobj = _shortcut_open(
  File "/home/shakti_gupta/anaconda3/lib/python3.9/site-packages/smart_open/smart_open_lib.py", line 361, in _shortcut_open
    return _builtin_open(local_path, mode, buffering=buffering, **open_kwargs)
FileNotFoundError: [Errno 2] No such file or directory: 'word2vec.model'

[merwok]

What build tool are you using? (setuptools or other) Are you configuring it properly to make it include the model file inside your sms_model package?

You should also move requirements.txt up one level, it does not belong in the python package. Not sure about the pickle file. These are not usually packaged, as the format changes between python versions.

[Shaktigupta931]

I am using setup tools, I have already moved requiremnet.txt to another folder and one level up but it is showing word2Vec.model and .pkl
FileNotFoundError: [Errno 2] No such file or directory: 'word2vec.model' while import my package in the pycharm

[merwok]

Are you configuring setuptools properly to make it include package data (the model file inside your sms_model package)?

[Shaktigupta931]

from setuptools import setup, find_packages setup( name="sms_module", version="0.0.1", packages=find_packages(), install_requires=[ "numpy", "pandas", "scikit-learn", "gensim" ], package_data={ 'sms_module': ['/home/shakti_gupta/Downloads/snsimpl/SmsModel/models/requirements.txt', '/home/shakti_gupta/Downloads/snsimpl/SmsModel/models/svc_classifier.pkl','/home/shakti_gupta/Downloads/snsimpl/SmsModel/models/word2vec.model'], } ) This is required setup.py files

[merwok]

The specification of package data is invalid, they should not be absolute paths.

See setuptools doc: https://setuptools.pypa.io/en/latest/userguide/quickstart.html#including-data-files

[Shaktigupta931]

_from setuptools import setup, find_packages setup( name="svc_model_flask", version="0.0.1", packages=find_packages(), install_requires=[ "numpy", "pandas", "scikit-learn", "gensim" ], data_files=[ ("/data", ["data/svcclassifier.pkl"]), ("/data", ["data/requirements.txt"]), ("/data", ["data/word2vec.model"]) ], ) I have updated the path but my code is not able to find the path model = Word2Vec.load("../data/word2vec.model") w2v_model = Word2Vec.load('../data/word2vec.model') with open('../data/svc_classifier.pkl', 'rb') as fid: svc_loaded = pickle.load(fid) Please let me know how can I load the path in my code after importing my whelll file

[merwok]

That’s using data files, which won’t work. You need to use package data. https://setuptools.pypa.io/en/latest/userguide/datafiles.html

[Shaktigupta931]

Resolved