Google Colab no longer runs Icevision

[Gytaco]

🐛 Bug

Describe the bug I was running models up through to end of June and early July in Google Colab with no issues. But now I get the following error on a simple pip icevision install.

OSError: /usr/local/lib/python3.7/dist-packages/torchtext/lib/libtorchtext.so: undefined symbol: _ZN3c1022getCustomClassTypeImplERKSt10type_index

To Reproduce Steps to reproduce the behavior:

1. Go to Google Colab and open a new notepad.
2. Click on settings and ensure its set to GPU.
3. add the following to your notebook. !pip install icevision[All]

from icevision.all import from icevision.models.utils import

5. Run those codes and see error.

Expected behavior This was working up till last month, so I know I'm pretty sure its a google colab issue, but I'm unsure how to fix the OS any help would be appreciated to remove this.

[prp0x80]

It's working fine on both colab and kaggle, I tried just now. There is no error while importing as you have mentioned. Although, there is one issue that appears later while fine tuning the model with learn.fine_tune(20, 0.00524, freeze_epochs=1)

NameError: Exception occured in `Recorder` when calling event `after_validate`:
    name 'create_coco_eval' is not defined

For some reason create_coco_eval function from convert_records_to_coco_style.py is not visible to coco_metric.py

[Gytaco]

Yes sorry I didn't revisit to update this.

As I was using FastAI to store the models there is some issue dependency issues between torch, latest fastai and icevision. I basically had to set the following package torchtext==0.9.1 and upgrade pillow to latest then I was no longer encountering any issues.

I will close this now.

[martimpassos]

Hi, I'm getting the same undefined symbol error upon import on Colab. If I introduce

!pip install torchtext==0.9.1
!pip install Pillow -U

as suggested above I get

/usr/local/lib/python3.7/dist-packages/torchvision/io/image.py:11: UserWarning: Failed to load image Python extension: /usr/local/lib/python3.7/dist-packages/torchvision/image.so: undefined symbol: _ZNK3c106IValue23reportToTensorTypeErrorEv
  warn(f"Failed to load image Python extension: {e}")

error and the session fails and restarts. Any clues?

[Gytaco]

Whats your entire library pip setup? I load icevision first then these are following with Fast ai last.

[martimpassos]

Whatever Colab's default is. I noticed it reinstalls torch some three times during the installs:

>>>!pip install icevision[all] icedata
>>>!pip install torchtext==0.9.1
>>>!pip install Pillow -U
>>>!pip freeze
absl-py==1.2.0
aeppl==0.0.33
aesara==2.7.9
aiohttp==3.8.1
aiosignal==1.2.0
alabaster==0.7.12
albumentations==1.0.3
altair==4.2.0
antlr4-python3-runtime==4.9.3
appdirs==1.4.4
arviz==0.12.1
astor==0.8.1
astropy==4.3.1
astunparse==1.6.3
async-timeout==4.0.2
asynctest==0.13.0
atari-py==0.2.9
atomicwrites==1.4.1
attrs==22.1.0
audioread==3.0.0
autograd==1.4
Babel==2.10.3
backcall==0.2.0
beautifulsoup4==4.6.3
bleach==5.0.1
blis==0.7.8
bokeh==2.3.3
branca==0.5.0
bs4==0.0.1
CacheControl==0.12.11
cached-property==1.5.2
cachetools==4.2.4
catalogue==2.0.8
certifi==2022.6.15
cffi==1.15.1
cftime==1.6.1
chardet==3.0.4
charset-normalizer==2.1.1
click==8.0.4
clikit==0.6.2
cloudpickle==1.5.0
cmake==3.22.6
cmdstanpy==1.0.7
colorcet==3.0.0
colorlover==0.3.0
community==1.0.0b1
cons==0.4.5
contextlib2==0.5.5
convertdate==2.4.0
crashtest==0.3.1
crcmod==1.7
cufflinks==0.17.3
cupy-cuda111==9.4.0
cvxopt==1.3.0
cvxpy==1.2.1
cycler==0.11.0
cymem==2.0.6
Cython==0.29.32
daft==0.0.4
dask==2022.2.0
dataclasses==0.6
datascience==0.17.5
debugpy==1.0.0
decorator==4.4.2
defusedxml==0.7.1
descartes==1.1.0
dill==0.3.5.1
distributed==2022.2.0
dlib==19.24.0
dm-tree==0.1.7
docker-pycreds==0.4.0
docutils==0.17.1
dopamine-rl==1.0.5
earthengine-api==0.1.323
easydict==1.9
ecos==2.0.10
editdistance==0.5.3
effdet==0.2.4
en-core-web-sm @ https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.4.0/en_core_web_sm-3.4.0-py3-none-any.whl
entrypoints==0.4
ephem==4.1.3
et-xmlfile==1.1.0
etils==0.7.1
etuples==0.3.8
fa2==0.3.5
fastai==2.5.6
fastcore==1.3.29
fastdownload==0.0.7
fastdtw==0.3.4
fastjsonschema==2.16.1
fastprogress==1.0.3
fastrlock==0.8
feather-format==0.4.1
filelock==3.8.0
fire==0.4.0
firebase-admin==4.4.0
fix-yahoo-finance==0.0.22
Flask==1.1.4
flatbuffers==2.0.7
folium==0.12.1.post1
frozenlist==1.3.1
fsspec==2022.8.2
future==0.16.0
fvcore==0.1.5.post20220512
gast==0.5.3
GDAL==2.2.2
gdown==4.4.0
gensim==3.6.0
geographiclib==1.52
geopy==1.17.0
gin-config==0.5.0
gitdb==4.0.9
GitPython==3.1.27
glob2==0.7
google==2.0.3
google-api-core==1.31.6
google-api-python-client==1.12.11
google-auth==1.35.0
google-auth-httplib2==0.0.4
google-auth-oauthlib==0.4.6
google-cloud-bigquery==1.21.0
google-cloud-bigquery-storage==1.1.2
google-cloud-core==1.0.3
google-cloud-datastore==1.8.0
google-cloud-firestore==1.7.0
google-cloud-language==1.2.0
google-cloud-storage==1.18.1
google-cloud-translate==1.5.0
google-colab @ file:///colabtools/dist/google-colab-1.0.0.tar.gz
google-pasta==0.2.0
google-resumable-media==0.4.1
googleapis-common-protos==1.56.4
googledrivedownloader==0.4
graphviz==0.10.1
greenlet==1.1.3
grpcio==1.48.1
gspread==3.4.2
gspread-dataframe==3.0.8
gym==0.25.2
gym-notices==0.0.8
h5py==3.1.0
HeapDict==1.0.1
hijri-converter==2.2.4
holidays==0.15
holoviews==1.14.9
html5lib==1.0.1
httpimport==0.5.18
httplib2==0.17.4
httplib2shim==0.0.3
httpstan==4.6.1
humanize==0.5.1
hyperopt==0.1.2
icedata==0.5.1
icevision==0.12.0
idna==2.10
imageio==2.9.0
imagesize==1.4.1
imbalanced-learn==0.8.1
imblearn==0.0
imgaug==0.4.0
importlib-metadata==4.12.0
importlib-resources==5.9.0
imutils==0.5.4
inflect==2.1.0
intel-openmp==2022.1.0
intervaltree==2.1.0
iopath==0.1.10
ipykernel==5.3.4
ipython==7.9.0
ipython-genutils==0.2.0
ipython-sql==0.3.9
ipywidgets==7.7.1
itsdangerous==1.1.0
jax==0.3.17
jaxlib @ https://storage.googleapis.com/jax-releases/cuda11/jaxlib-0.3.15+cuda11.cudnn805-cp37-none-manylinux2014_x86_64.whl
jedi==0.18.1
jieba==0.42.1
Jinja2==2.11.3
joblib==1.1.0
jpeg4py==0.1.4
jsonschema==4.3.3
jupyter-client==6.1.12
jupyter-console==6.1.0
jupyter-core==4.11.1
jupyterlab-widgets==3.0.3
kaggle==1.5.12
kapre==0.3.7
keras==2.8.0
Keras-Preprocessing==1.1.2
keras-vis==0.4.1
kiwisolver==1.4.4
korean-lunar-calendar==0.2.1
langcodes==3.3.0
libclang==14.0.6
librosa==0.8.1
lightgbm==2.2.3
llvmlite==0.39.1
lmdb==0.99
locket==1.0.0
logical-unification==0.4.5
loguru==0.6.0
LunarCalendar==0.0.9
lxml==4.9.1
Markdown==3.4.1
MarkupSafe==2.0.1
marshmallow==3.17.1
matplotlib==3.2.2
matplotlib-venn==0.11.7
miniKanren==1.0.3
missingno==0.5.1
mistune==0.8.4
mizani==0.7.3
mkl==2019.0
mlxtend==0.14.0
more-itertools==8.14.0
moviepy==0.2.3.5
mpmath==1.2.1
msgpack==1.0.4
multidict==6.0.2
multipledispatch==0.6.0
multitasking==0.0.11
murmurhash==1.0.8
music21==5.5.0
natsort==5.5.0
nbconvert==5.6.1
nbformat==5.4.0
netCDF4==1.6.0
networkx==2.6.3
nibabel==3.0.2
nltk==3.7
nose==1.3.7
notebook==5.3.1
numba==0.56.2
numexpr==2.8.3
numpy==1.21.6
oauth2client==4.1.3
oauthlib==3.2.0
okgrade==0.4.3
omegaconf==2.2.3
opencv-contrib-python==4.6.0.66
opencv-python==4.6.0.66
opencv-python-headless==4.6.0.66
openpyxl==3.0.10
opt-einsum==3.3.0
osqp==0.6.2.post0
packaging==21.3
palettable==3.3.0
pandas==1.3.5
pandas-datareader==0.9.0
pandas-gbq==0.13.3
pandas-profiling==1.4.1
pandocfilters==1.5.0
panel==0.12.1
param==1.12.2
parso==0.8.3
partd==1.3.0
pastel==0.2.1
pathlib==1.0.1
pathtools==0.1.2
pathy==0.6.2
patsy==0.5.2
pep517==0.13.0
pexpect==4.8.0
pickleshare==0.7.5
Pillow==9.2.0
pip-tools==6.2.0
plotly==5.5.0
plotnine==0.8.0
pluggy==0.7.1
pooch==1.6.0
portalocker==2.5.1
portpicker==1.3.9
prefetch-generator==1.0.1
preshed==3.0.7
prettytable==3.4.1
progressbar2==3.38.0
promise==2.3
prompt-toolkit==2.0.10
prophet==1.1
protobuf==3.17.3
psutil==5.4.8
psycopg2==2.9.3
ptyprocess==0.7.0
py==1.11.0
pyarrow==6.0.1
pyasn1==0.4.8
pyasn1-modules==0.2.8
pybboxes==0.1.4
pycocotools==2.0.4
pycparser==2.21
pyct==0.4.8
pydantic==1.9.2
pydata-google-auth==1.4.0
pyDeprecate==0.3.2
pydot==1.3.0
pydot-ng==2.0.0
pydotplus==2.0.2
PyDrive==1.3.1
pyemd==0.5.1
pyerfa==2.0.0.1
Pygments==2.6.1
pygobject==3.26.1
pylev==1.4.0
pymc==4.1.4
PyMeeus==0.5.11
pymongo==4.2.0
pymystem3==0.2.0
PyOpenGL==3.1.6
pyparsing==3.0.9
pyrsistent==0.18.1
pysimdjson==3.2.0
pysndfile==1.3.8
PySocks==1.7.1
pystan==3.3.0
pytest==3.6.4
python-apt==0.0.0
python-chess==0.23.11
python-dateutil==2.8.2
python-louvain==0.16
python-slugify==6.1.2
python-utils==3.3.3
pytorch-lightning==1.7.6
pytz==2022.2.1
pyviz-comms==2.2.1
PyWavelets==1.3.0
PyYAML==6.0
pyzmq==23.2.1
qdldl==0.1.5.post2
qudida==0.0.4
regex==2022.6.2
requests==2.23.0
requests-oauthlib==1.3.1
resampy==0.4.0
resnest==0.0.6b20220916
rpy2==3.4.5
rsa==4.9
sahi==0.10.5
scikit-image==0.18.3
scikit-learn==1.0.2
scipy==1.7.3
screen-resolution-extra==0.0.0
scs==3.2.0
seaborn==0.11.2
Send2Trash==1.8.0
sentry-sdk==1.9.0
setproctitle==1.3.2
setuptools-git==1.2
Shapely==1.8.4
shortuuid==1.0.9
six==1.15.0
sklearn-pandas==1.8.0
smart-open==5.2.1
smmap==5.0.0
snowballstemmer==2.2.0
sortedcontainers==2.4.0
SoundFile==0.10.3.post1
spacy==3.4.1
spacy-legacy==3.0.10
spacy-loggers==1.0.3
Sphinx==1.8.6
sphinxcontrib-serializinghtml==1.1.5
sphinxcontrib-websupport==1.2.4
SQLAlchemy==1.4.41
sqlparse==0.4.2
srsly==2.4.4
statsmodels==0.12.2
sympy==1.7.1
tables==3.7.0
tabulate==0.8.10
tblib==1.7.0
tenacity==8.0.1
tensorboard==2.10.0
tensorboard-data-server==0.6.1
tensorboard-plugin-wit==1.8.1
tensorflow==2.8.2+zzzcolab20220719082949
tensorflow-datasets==4.6.0
tensorflow-estimator==2.8.0
tensorflow-gcs-config==2.8.0
tensorflow-hub==0.12.0
tensorflow-io-gcs-filesystem==0.26.0
tensorflow-metadata==1.10.0
tensorflow-probability==0.16.0
termcolor==1.1.0
terminado==0.13.3
terminaltables==3.1.10
testpath==0.6.0
text-unidecode==1.3
textblob==0.15.3
thinc==8.1.0
threadpoolctl==3.1.0
tifffile==2021.11.2
timm==0.6.7
toml==0.10.2
tomli==2.0.1
toolz==0.12.0
torch==1.8.1
torchaudio @ https://download.pytorch.org/whl/cu113/torchaudio-0.12.1%2Bcu113-cp37-cp37m-linux_x86_64.whl
torchmetrics==0.9.3
torchsummary==1.5.1
torchtext==0.9.1
torchvision==0.11.3
tornado==5.1.1
tqdm==4.64.1
traitlets==5.1.1
tweepy==3.10.0
typeguard==2.7.1
typer==0.4.2
typing-extensions==4.1.1
tzlocal==1.5.1
ujson==5.4.0
uritemplate==3.0.1
urllib3==1.24.3
vega-datasets==0.9.0
wandb==0.13.3
wasabi==0.10.1
wcwidth==0.2.5
webargs==8.2.0
webencodings==0.5.1
Werkzeug==1.0.1
widgetsnbextension==3.6.1
wordcloud==1.8.2.2
wrapt==1.14.1
xarray==0.20.2
xarray-einstats==0.2.2
xgboost==0.90
xkit==0.0.0
xlrd==1.1.0
xlwt==1.3.0
yacs==0.1.8
yarl==1.8.1
yellowbrick==1.5
yolov5-icevision==6.0.0
zict==2.2.0
zipp==3.8.1

[Gytaco]

Ahh I just reread your error, it is on torchvision where as the issue I was experiencing is on torchtext.

I originally thought it was Colab too, but its just the upgrades being made to the other dependencies icevision relies on.

My guess is you need to play around with the versions to get the right setup that will work with Icevision with your setup. Sorry that doesn't solve the issue for you.