google-research / text-to-text-transfer-transformer

Code for the paper "Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer"
https://arxiv.org/abs/1910.10683
Apache License 2.0
6.12k stars 753 forks source link

Errors in Deploy Notebook #970

Open peregilk opened 2 years ago

peregilk commented 2 years ago

When running the deploy Notebook, I keep getting this error:

WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow/python/compat/v2_compat.py:111: disable_resource_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in a future version.
Instructions for updating:
non-resource variables are not supported in the long term
I0201 18:45:59.303558 140008391333760 mesh_transformer_main.py:172] No write access to model directory. Skipping command logging.
W0201 18:45:59.309430 140008391333760 utils.py:116] Could not extract mixture/task name from gin config.
W0201 18:45:59.309546 140008391333760 utils.py:133] Using default vocabulary.
I0201 18:45:59.657206 140008391333760 resource_reader.py:50] system_path_file_exists:gs://t5-data/pretrained_models/cbqa/small_ssm_nq/operative_config.gin
E0201 18:45:59.657829 140008391333760 resource_reader.py:55] Path not found: gs://t5-data/pretrained_models/cbqa/small_ssm_nq/operative_config.gin
I0201 18:45:59.951137 140008391333760 config.py:2372] Skipping import of unknown module `t5.data.sentencepiece_vocabulary` (skip_unknown=True).
Traceback (most recent call last):

I am getting the same errors trying to do this from the command line, but reproducing in the Notebook is easier.

peregilk commented 2 years ago

Thanks @adarob for trying to fix this.

However, I am still unable to run this notebook. For some reason (even if the google-research clone seem to be OK). It is still unable to import the module.

I am getting:

ModuleNotFoundError: No module named 't5_cbqa'
peregilk commented 2 years ago

@adarob I have partly a workaround here, so updating the post.

First, I am getting these errors when installing t5 in my Colab

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
yellowbrick 1.3.post1 requires numpy<1.20,>=1.16.0, but you have numpy 1.21.5 which is incompatible.
datascience 0.10.6 requires folium==0.2.1, but you have folium 0.8.3 which is incompatible.
albumentations 0.1.12 requires imgaug<0.2.7,>=0.2.5, but you have imgaug 0.2.9 which is incompatible.

But the install continues despite these errors.

Next cell crashes complaining first about a missing folder, then about the missing module. I am able to get around this by adding this code to the second cell:

if not os.path.isdir(saved_model_dir):
  os.mkdir(saved_model_dir)

%env PYTHONPATH="/content/google-research/:/content/google-research/t5_closed_book_qa:${PYTHONPATH}"

Now the model is saved (For my use case this is sufficient). However, the next cell crashes with the following error:

RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd

As you see from the pip install log at the top of this posts, there are some numpy-version conflicts. My guess is that this is related but I have not digged any deeper into this.

adarob commented 2 years ago

Sorry about that. I'm adding in your python path fix. The second error seems more like a Colab bug.