search

gramineproject / examples

Sample applications configs for Gramine
BSD 3-Clause "New" or "Revised" License
29 stars 22 forks source link

Pytorch example not working with newly released Pillow10.4.0 and default numpy 1.20.1 for RPM distros #102

Open jinengandhi-intel opened 6 days ago

jinengandhi-intel commented 6 days ago

On July 1st a new version of Pillow 10.4.0 (https://pypi.org/project/pillow/) was released which is pulled automatically by all RPM based distros like CentOS/RHEL/Rocky Linux/Alma Linux resulting in a failure in the local CI nightly:

python3 download-pretrained-model.py
Traceback (most recent call last):
  File "/home/intel/jenkins/workspace/local_ci_graphene_native_rhel9_server_6.7/gramine/CI-Examples/pytorch/download-pretrained-model.py", line 3, in <module>
    from torchvision import models
  File "/usr/local/lib64/python3.9/site-packages/torchvision/__init__.py", line 6, in <module>
    from torchvision import _meta_registrations, datasets, io, models, ops, transforms, utils
  File "/usr/local/lib64/python3.9/site-packages/torchvision/datasets/__init__.py", line 1, in <module>
    from ._optical_flow import FlyingChairs, FlyingThings3D, HD1K, KittiFlow, Sintel
  File "/usr/local/lib64/python3.9/site-packages/torchvision/datasets/_optical_flow.py", line 10, in <module>
    from PIL import Image
  File "/usr/local/lib64/python3.9/site-packages/PIL/Image.py", line 68, in <module>
    from ._typing import StrOrBytesPath, TypeGuard
  File "/usr/local/lib64/python3.9/site-packages/PIL/_typing.py", line 10, in <module>
    NumpyArray = npt.NDArray[Any]
AttributeError: module 'numpy.typing' has no attribute 'NDArray'

After some more debugging we found out that the numpy version that is pulled by default is 1.20.1 and the combination of this version with the new Pillow version is what is causing the error.

When we upgrade numpy to any version >1.21.5 things again work fine and we don't see any error.

Debian based distros are still tagged to older versions of Pillow and hence no regressions are seen for those distros.

Current README for pytorch has steps which have been tested for 18.04 which is very old. Should we update that and as part of list the versions of different components?

dimakuv commented 3 days ago

After some more debugging we found out that the numpy version that is pulled by default is 1.20.1

What do you mean by "pulled by default". Pulled by whom, by the OS distro? Or by our Makefiles / README steps?

Current README for pytorch has steps which have been tested for 18.04 which is very old. Should we update that and as part of list the versions of different components?

Yes, let's do it. Could you create such a PR?