search

hezarai / hezar

The all-in-one AI library for Persian, supporting a wide variety of tasks and modalities!
https://hezarai.github.io/hezar/
Apache License 2.0
822 stars 45 forks source link

Set up lazy loading for it installs so many libraries right now #66

Closed pooya-mohammadi closed 10 months ago

arxyzan commented 10 months ago

The main issue here is that the whole registry system works only when modules are actually imported. How can we handle this?

arxyzan commented 10 months ago

The latest commits for v0.26.0 is somehow dedicated to this task. Setting up lazy imports the way Transformers does it seems to be overkill for us right now. In our new implemented schema there is a new class variable in all main modules called required_backends which is a list of libraries that are required for the module to work. Right now some additional libraries like librosa, soundfile, pillow, nltk, etc. are not installed along with Hezar as they used to but instead, they are optional. The users will be notified of any unavailable package only if they use a class that necessarily needs it.

For example if you run the follwing code:

from hezar import Model
model = Model.load("hezarai/trocr-fa-v1")

The following error will be raised:

ModuleNotFoundError: `TrOCRImage2Text` requires `PIL` which is not installed!