search

pythonlessons / mltu

Machine Learning Training Utilities (for TensorFlow and PyTorch)
MIT License
160 stars 100 forks source link

Tutorial 2 - ModuleNotFoundError: No module named 'mltu' #21

Closed goodwin64 closed 9 months ago

goodwin64 commented 11 months ago

Heyya! Thanks for a cool repo and intro video explaining it. I'm particularly interested in the 2nd tutorial - captcha text recognition.

I'm new to the python ecosystem so sorry in advance for silly questions. After cloning the repo and installing the dependencies via:

pip install -r requirements.txt

I get the following error:

maxdonchenko@maxdonchenko mltu % python3 ./Tutorials/02_captcha_to_text/train.py
Traceback (most recent call last):
  File "/Users/maxdonchenko/mltu/./Tutorials/02_captcha_to_text/train.py", line 7, in <module>
    from mltu.tensorflow.dataProvider import DataProvider
ModuleNotFoundError: No module named 'mltu'

An interesting nuance here is that I think I have an mltu installed:

maxdonchenko@maxdonchenko mltu % pip show mltu
Name: mltu
Version: 1.0.15
Summary: Machine Learning Training Utilities (MLTU) for TensorFlow and PyTorch
Home-page: https://pylessons.com/
Author: PyLessons
Author-email: pythonlessons0@gmail.com
License: 
Location: /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages
Requires: librosa, matplotlib, numpy, onnxruntime, opencv-python, pandas, Pillow, PyYAML, tqdm
Required-by:

I tried to explicitly specify the mltu version in the root requirements.txt:

PyYAML>=6.0
tqdm
pandas
numpy
opencv-python
Pillow>=9.4.0
onnxruntime>=1.15.0  # onnxruntime-gpu for GPU support
librosa>=0.9.2
matplotlib
# 👆 already existing in the repo
# 👇 added by me
mltu==0.1.4
tensorflow==2.10 # took versions from Tutorials/02_captcha_to_text/README.md

but, installing them threw another error, basically saying tf version 2.10 can't be installed for some reason:

ERROR: Could not find a version that satisfies the requirement tensorflow==2.10 (from versions: 2.13.0rc0, 2.13.0rc1, 2.13.0rc2, 2.13.0)
ERROR: No matching distribution found for tensorflow==2.10

Do you have any suggestions on how to get the same environment that you had during NN model training?

pythonlessons commented 11 months ago

As I tested, with tensorflow 2.13 export to onnx model doesn't work for some reason. What Python version you use?

Have you tried to do "import mltu" after installing it?

goodwin64 commented 11 months ago

with tensorflow 2.13 export to onnx model doesn't work for some reason.

I noticed an old distributive of the TensorFlow (e.g. 2.10) isn't available to download - maybe TF contributors delete it after some time.

What Python version you use?

@maxdonchenko mltu % python3 --version
Python 3.11.4

Have you tried to do "import mltu" after installing it?

Not sure I know exactly how to do it but if I add it to the top of Tutorials/02_captcha_to_text/train.py:

import mltu # <--- added
import tensorflow as tf
try: [tf.config.experimental.set_memory_growth(gpu, True) for gpu in tf.config.experimental.list_physical_devices("GPU")]
except: pass

then it doesn't work:

@maxdonchenko mltu % python3 Tutorials/02_captcha_to_text/train.py 
Traceback (most recent call last):
  File "/Users/maxdonchenko/mltu/Tutorials/02_captcha_to_text/train.py", line 1, in <module>
    import mltu
ModuleNotFoundError: No module named 'mltu'
pythonlessons commented 11 months ago

That's really strange, is it possible to try this stuff with 3.10 Python for you?

goodwin64 commented 11 months ago

If I downgrade my python version to 3.10 (via pyenv):

maxdonchenko@maxdonchenko mltu % python -V
Python 3.10.12
maxdonchenko@maxdonchenko mltu % pip -V   
pip 23.0.1 from /Users/maxdonchenko/.pyenv/versions/3.10.12/lib/python3.10/site-packages/pip (python 3.10)

and want to install this pair:

# requirements.txt
# some other dependencies...
tensorflow==2.10
mltu==0.1.4

then I get:

maxdonchenko@maxdonchenko mltu % pip install -r requirements.txt 
Collecting PyYAML>=6.0
  Downloading PyYAML-6.0.1-cp310-cp310-macosx_11_0_arm64.whl (169 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 169.3/169.3 kB 1.9 MB/s eta 0:00:00
Collecting tqdm
  Using cached tqdm-4.65.0-py3-none-any.whl (77 kB)
Collecting pandas
  Downloading pandas-2.0.3-cp310-cp310-macosx_11_0_arm64.whl (10.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.8/10.8 MB 9.7 MB/s eta 0:00:00
Collecting numpy
  Downloading numpy-1.25.2-cp310-cp310-macosx_11_0_arm64.whl (14.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 14.0/14.0 MB 2.9 MB/s eta 0:00:00
Collecting opencv-python
  Using cached opencv_python-4.8.0.74-cp37-abi3-macosx_11_0_arm64.whl (33.1 MB)
Collecting Pillow>=9.4.0
  Downloading Pillow-10.0.0-cp310-cp310-macosx_11_0_arm64.whl (3.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.1/3.1 MB 1.7 MB/s eta 0:00:00
Collecting onnxruntime>=1.15.0
  Downloading onnxruntime-1.15.1-cp310-cp310-macosx_11_0_arm64.whl (6.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.1/6.1 MB 1.7 MB/s eta 0:00:00
Collecting librosa>=0.9.2
  Using cached librosa-0.10.0.post2-py3-none-any.whl (253 kB)
Collecting matplotlib
  Downloading matplotlib-3.7.2-cp310-cp310-macosx_11_0_arm64.whl (7.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.3/7.3 MB 2.1 MB/s eta 0:00:00
ERROR: Could not find a version that satisfies the requirement tensorflow==2.10 (from versions: 2.13.0rc0, 2.13.0rc1, 2.13.0rc2, 2.13.0)
ERROR: No matching distribution found for tensorflow==2.10

[notice] A new release of pip is available: 23.0.1 -> 23.2.1
[notice] To update, run: pip install --upgrade pip

if then I try tf version 2.13:

# requirements.txt
# some other dependencies...
tensorflow==2.13
mltu==0.1.4

I get the same error as before with higher version of python:

maxdonchenko@maxdonchenko mltu % pip install -r requirements.txt
Collecting ... (lots of libraries here)
  Using cached ...whl (451 kB)

ERROR: Cannot install -r requirements.txt (line 11) and tensorflow==2.13 because these package versions have conflicting dependencies.

The conflict is caused by:
    The user requested tensorflow==2.13
    mltu 0.1.4 depends on tensorflow<=2.10.1

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts

Aaaaand.. the option 3 - to omit versions at all:

# requirements.txt
tensorflow
mltu

will install mltu-1.0.15 and tensorflow-2.13.0. I don't get installation errors but the training is failing then:

maxdonchenko@maxdonchenko mltu % python Tutorials/02_captcha_to_text/train.py
Traceback (most recent call last):
  File "/Users/maxdonchenko/mltu/Tutorials/02_captcha_to_text/train.py", line 9, in <module>
    from mltu.tensorflow.callbacks import Model2onnx, TrainLogger
  File "/Users/maxdonchenko/.pyenv/versions/3.10.12/lib/python3.10/site-packages/mltu/tensorflow/callbacks.py", line 6, in <module>
    class Model2onnx(Callback):
  File "/Users/maxdonchenko/.pyenv/versions/3.10.12/lib/python3.10/site-packages/mltu/tensorflow/callbacks.py", line 9, in Model2onnx
    import onnx
ModuleNotFoundError: No module named 'onnx'
pythonlessons commented 11 months ago

yes, you need to install onnx and tf2onnx packages, but with 2.13 TensorFlow version it will crash either, I suggest 2.12 And, yes, use newest mltu version, as I try to solve all bugs

goodwin64 commented 11 months ago

Probably the "mission is impossible" here 😅

As TensorFlow <2.13 is unavailable:

tensorflow==2.12
mltu
onnx
tf2onnx
maxdonchenko@maxdonchenko mltu % pip install -r requirements.txt
ERROR: Could not find a version that satisfies the requirement tensorflow==2.12 (from versions: 2.13.0rc0, 2.13.0rc1, 2.13.0rc2, 2.13.0)
ERROR: No matching distribution found for tensorflow==2.12

while v 2.13 throws the following error:

maxdonchenko@maxdonchenko mltu % python Tutorials/02_captcha_to_text/train.py
/Users/maxdonchenko/.pyenv/versions/3.10.12/lib/python3.10/site-packages/tf2onnx/utils.py:46: FutureWarning: In the future `np.bool` will be defined as the corresponding NumPy scalar.
  onnx_pb.TensorProto.BOOL: np.bool,
Traceback (most recent call last):
  File "/Users/maxdonchenko/mltu/Tutorials/02_captcha_to_text/train.py", line 9, in <module>
    from mltu.tensorflow.callbacks import Model2onnx, TrainLogger
  File "/Users/maxdonchenko/.pyenv/versions/3.10.12/lib/python3.10/site-packages/mltu/tensorflow/callbacks.py", line 6, in <module>
    class Model2onnx(Callback):
  File "/Users/maxdonchenko/.pyenv/versions/3.10.12/lib/python3.10/site-packages/mltu/tensorflow/callbacks.py", line 10, in Model2onnx
    import tf2onnx
  File "/Users/maxdonchenko/.pyenv/versions/3.10.12/lib/python3.10/site-packages/tf2onnx/__init__.py", line 15, in <module>
    from tf2onnx import tfonnx, utils, graph, graph_builder, graph_matcher, shape_inference, schemas, convert  # pylint: disable=wrong-import-order
  File "/Users/maxdonchenko/.pyenv/versions/3.10.12/lib/python3.10/site-packages/tf2onnx/tfonnx.py", line 20, in <module>
    import tf2onnx.onnx_opset  # pylint: disable=unused-import
  File "/Users/maxdonchenko/.pyenv/versions/3.10.12/lib/python3.10/site-packages/tf2onnx/onnx_opset/__init__.py", line 5, in <module>
    from . import (
  File "/Users/maxdonchenko/.pyenv/versions/3.10.12/lib/python3.10/site-packages/tf2onnx/onnx_opset/controlflow.py", line 19, in <module>
    from tf2onnx import utils
  File "/Users/maxdonchenko/.pyenv/versions/3.10.12/lib/python3.10/site-packages/tf2onnx/utils.py", line 46, in <module>
    onnx_pb.TensorProto.BOOL: np.bool,
  File "/Users/maxdonchenko/.pyenv/versions/3.10.12/lib/python3.10/site-packages/numpy/__init__.py", line 305, in __getattr__
    raise AttributeError(__former_attrs__[attr])
AttributeError: module 'numpy' has no attribute 'bool'.
`np.bool` was a deprecated alias for the builtin `bool`. To avoid this error in existing code, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.
The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at:
    https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations. Did you mean: 'bool_'?

Do you know any context on why the TensorFlow team removes previous versions of distributive?

dragonsxslayer commented 10 months ago

i have same problem in windows wsl :(

pythonlessons commented 10 months ago

Yes, windows require WSL from TensorFlow 2.11, so it's complicated. This is why I recommend using Python 3.10 with TF 2.10 Or move to PyTorch