Problems importing Tensorflow from installer

[mzieg]

Simplest version of the problem. This works:

Z:\code\ENLIGHTEN>type tf.py
import tensorflow as tf
print(tf.reduce_sum(tf.random.normal([1000, 1000])))

Z:\code\ENLIGHTEN>python tf.py
tf.Tensor(498.73596, shape=(), dtype=float32)

This seems to work:

Z:\code\ENLIGHTEN>pyinstaller --distpath=dist --workpath=work --exclude-module=_bootlocale tf.py

This fails:

Z:\code\ENLIGHTEN> dist\tf\tf.exe
Traceback (most recent call last):
  File "\\Mac\work\code\ENLIGHTEN\tf.py", line 1, in <module>
    import tensorflow as tf
  File "PyInstaller\loader\pyimod03_importers.py", line 546, in exec_module
  File "tensorflow\__init__.py", line 38, in <module>
  File "PyInstaller\loader\pyimod03_importers.py", line 546, in exec_module
  File "tensorflow\python\__init__.py", line 42, in <module>
  File "PyInstaller\loader\pyimod03_importers.py", line 546, in exec_module
  File "tensorflow\python\saved_model\saved_model.py", line 20, in <module>
  File "PyInstaller\loader\pyimod03_importers.py", line 546, in exec_module
  File "tensorflow\python\saved_model\builder.py", line 23, in <module>
  File "PyInstaller\loader\pyimod03_importers.py", line 546, in exec_module
  File "tensorflow\python\saved_model\builder_impl.py", line 26, in <module>
  File "PyInstaller\loader\pyimod03_importers.py", line 546, in exec_module
  File "tensorflow\python\framework\dtypes.py", line 28, in <module>
ImportError: DLL load failed while importing _dtypes: The specified module could not be found.
[1912] Failed to execute script 'tf' due to unhandled exception!

[mzieg]

I think this is converging, but want to leave some breadcrumbs as I need to reboot. This works better, or at least seems to advance to a better quality of bug:

$ cat tf.py
import ml_dtypes 
import tensorflow as tf
import tensorflow.python.framework.dtypes as dt

print(f"ml_dtypes is a {ml_dtypes}")
print(f"dt is a {dt}")
print(tf.reduce_sum(tf.random.normal([1000, 1000])))

I'm currently using the following build command, but I'm sure it can be trimmed:

pyinstaller ^
  --distpath=dist ^
  --workpath=work ^
  --add-data="c:\Users\mzieg\miniconda3\envs\conda_enlighten3\Lib\site-packages\tensorflow;." ^
  --add-data="c:\Users\mzieg\miniconda3\envs\conda_enlighten3\Lib\site-packages\tensorflow\python;." ^
  --hidden-import="ml_dtypes" ^
  --hidden-import="tensorflow.python.pywrap_tensorflow_internal" ^
  --hidden-import="tensorflow.python._pywrap_tensorflow_internal" ^
  --exclude-module=_bootlocale ^
  tf.py

[mzieg]

For the record, this works fine on Python 3.11. I think we need to port from PySide2 to PySide6 to use that :-(

C:\Users\mzieg\test>type environment.yml
name: conda_tf

channels:
     - defaults
     - conda-forge

dependencies:
     - numpy
     - pandas
     - pexpect
     - pip
     - psutil
     - python
     - pytest
     - scipy
     - xlwt
     - pywin32

C:\Users\mzieg\test>pip list
Package                      Version
---------------------------- ------------
absl-py                      2.0.0
altgraph                     0.17.4
astunparse                   1.6.3
Bottleneck                   1.3.5
cachetools                   5.3.1
certifi                      2023.7.22
charset-normalizer           3.3.0
colorama                     0.4.6
flatbuffers                  23.5.26
gast                         0.5.4
google-auth                  2.23.2
google-auth-oauthlib         1.0.0
google-pasta                 0.2.0
grpcio                       1.59.0
h5py                         3.9.0
idna                         3.4
iniconfig                    1.1.1
keras                        2.14.0
libclang                     16.0.6
Markdown                     3.4.4
MarkupSafe                   2.1.3
mkl-fft                      1.3.8
mkl-random                   1.2.4
mkl-service                  2.4.0
ml-dtypes                    0.2.0
numexpr                      2.8.4
numpy                        1.26.0
oauthlib                     3.2.2
opt-einsum                   3.3.0
packaging                    23.1
pandas                       2.0.3
pefile                       2023.2.7
pexpect                      4.8.0
pip                          23.2.1
pip-system-certs             4.0
pluggy                       1.0.0
protobuf                     4.24.3
psutil                       5.9.0
ptyprocess                   0.7.0
pyasn1                       0.5.0
pyasn1-modules               0.3.0
pyinstaller                  6.0.0
pyinstaller-hooks-contrib    2023.9
pytest                       7.4.0
python-dateutil              2.8.2
pytz                         2023.3.post1
pywin32                      305.1
pywin32-ctypes               0.2.2
requests                     2.31.0
requests-oauthlib            1.3.1
rsa                          4.9
scipy                        1.11.1
setuptools                   68.0.0
six                          1.16.0
tensorboard                  2.14.1
tensorboard-data-server      0.7.1
tensorflow                   2.14.0
tensorflow-estimator         2.14.0
tensorflow-intel             2.14.0
tensorflow-io-gcs-filesystem 0.31.0
termcolor                    2.3.0
typing_extensions            4.8.0
tzdata                       2023.3
urllib3                      2.0.5
Werkzeug                     2.3.7
wheel                        0.41.2
wrapt                        1.14.1
xlwt                         1.3.0

C:\Users\mzieg\test>type make-tf.bat
@echo off

echo %time% purging old
rm -rfv tf-work tf-dist > rm.out

echo %time% running pyinstaller
pyinstaller ^
  --distpath=tf-dist ^
  --workpath=tf-work ^
  --exclude-module=_bootlocale ^
  tf.py

echo %time% running tf...
tf-dist\tf\tf

echo %time% done

C:\Users\mzieg\test>type tf.py
import pip_system_certs.wrapt_requests as wr
print(f"wr is {wr}\n")

import importlib
print(f"importlib is {importlib}\n")

import ml_dtypes
print(f"ml_dtypes is {ml_dtypes}\n")

import tensorflow.python.keras.callbacks as kcb
print(f"kcb is {kcb}\n")

from tensorflow.python.framework.tensor_util import constant_value as get_static_value
print(f"get_static_value is {get_static_value}\n")

from tensorflow.python.tpu import api
print(f"api is {api}\n")

from tensorflow.python.tools import module_util as _module_util
print(f"_module_util is {_module_util}\n")

import tensorflow.python.framework.dtypes as dt
print(f"dt is {dt}\n")

import tensorflow as tf
print(f"tf is {tf}\n")

print(tf.reduce_sum(tf.random.normal([1000, 1000])))

C:\Users\mzieg\test>tf-dist\tf\tf
wr is <module 'pip_system_certs.wrapt_requests' from 'C:\\Users\\mzieg\\test\\tf-dist\\tf\\_internal\\pip_system_certs\\wrapt_requests.pyc'>

importlib is <module 'importlib' from 'C:\\Users\\mzieg\\test\\tf-dist\\tf\\_internal\\importlib\\__init__.pyc'>

ml_dtypes is <module 'ml_dtypes' from 'C:\\Users\\mzieg\\test\\tf-dist\\tf\\_internal\\ml_dtypes\\__init__.pyc'>

WARNING:tensorflow:AutoGraph is not available in this environment: functions lack code information. This is typical of some environments like the interactive Python shell. See https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/autograph/g3doc/reference/limitations.md#access-to-source-code for more information.
kcb is <module 'tensorflow.python.keras.callbacks' from 'C:\\Users\\mzieg\\test\\tf-dist\\tf\\_internal\\tensorflow\\python\\keras\\callbacks.pyc'>

get_static_value is <function constant_value at 0x00000206C1C4D8A0>

api is <module 'tensorflow.python.tpu.api' from 'C:\\Users\\mzieg\\test\\tf-dist\\tf\\_internal\\tensorflow\\python\\tpu\\api.pyc'>

_module_util is <module 'tensorflow.python.tools.module_util' from 'C:\\Users\\mzieg\\test\\tf-dist\\tf\\_internal\\tensorflow\\python\\tools\\module_util.pyc'>

dt is <module 'tensorflow.python.framework.dtypes' from 'C:\\Users\\mzieg\\test\\tf-dist\\tf\\_internal\\tensorflow\\python\\framework\\dtypes.pyc'>

tf is <module 'tensorflow' from 'C:\\Users\\mzieg\\test\\tf-dist\\tf\\_internal\\tensorflow\\__init__.pyc'>

2023-09-30 09:42:26.225562: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: SSE SSE2 SSE3 SSE4.1 SSE4.2 AVX AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
tf.Tensor(2039.2915, shape=(), dtype=float32)

[mzieg]

If we want to explore Python 3.11, we need to explore PySide6, so some notes down that tree.

So exciting that these are visibly different sizes yet report the same version:

-rwxr-xr-x 1 mzieg 197121 137872 Sep 30 10:42 ./envs/conda_enlighten3/Lib/site-packages/PySide6/rcc.exe*
-rwxr-xr-x 1 mzieg 197121 108420 Sep 30 10:43 ./envs/conda_enlighten3/Scripts/pyside6-rcc.exe*

mzieg@MARKZIEGE8D0 MINGW64 ~/miniconda3
$ ./envs/conda_enlighten3/Lib/site-packages/PySide6/rcc.exe --version
rcc 6.5.2

mzieg@MARKZIEGE8D0 MINGW64 ~/miniconda3
$ ./envs/conda_enlighten3/Scripts/pyside6-rcc.exe --version
rcc 6.5.2

[mzieg]

Given all the fun I'm having with conda, this is at least encouraging: https://www.anaconda.com/blog/a-faster-conda-for-a-growing-community

[mzieg]

The port to PySide6 and Python 3.11 was pretty painless. We're past the "module not found" import errors, and are now at an arguably more solvable-looking bug (still tied to import, though).

Googling suggests this tends to mean "try a different numpy version", and I've tried 1.24, 1.25 and 1.26 (released last week). Maybe time to back down to an earlier Tensorflow again?

[mzieg]

Keeping numpy 1.26 but rolling back to Tensorflow 2.13 (all on Python 3.11) seemed to do the trick

[mzieg]

resolved by #304