Tensorflow model analysis on datalab example

[OrielResearchCure]

Hello,

Are you familiar with example for tensorflow model analysis on datalab. For example: https://www.tensorflow.org/tfx/model_analysis/#getting_started

Thanks, eilalan

[OrielResearchCure]

Quick update on my trials, in case anyone has advise for me:

!pip install -U tensorflow
import tensorflow as tf
!pip install tensorflow_model_analysis
import tensorflow_model_analysis as tfma

This is the import error:

ImportErrorTraceback (most recent call last)
<ipython-input-17-f85e4d8fbd99> in <module>()
----> 1 import tensorflow_model_analysis as tfma

/usr/local/envs/py2env/lib/python2.7/site-packages/tensorflow_model_analysis/__init__.py in <module>()
     15 
     16 
---> 17 from tensorflow_model_analysis import view
     18 from tensorflow_model_analysis.api import tfma_unit as test
     19 from tensorflow_model_analysis.api.model_eval_lib import *  # pylint: disable=wildcard-import

ImportError: cannot import name view

Any idea? Thanks, eilalan

[paveldournov]

Please use the explicit python 2 environment

%%bash
source activate py2env
pip install ... 

[OrielResearchCure]

thanks. I was able to make it work by working with python 2. What will be the substitution for enabling these widgets?

jupyter nbextension enable --py widgetsnbextension jupyter nbextension install --py --symlink tensorflow_model_analysis jupyter nbextension enable --py tensorflow_model_analysis

Thanks, eilalan

[OrielResearchCure]

BTW, another issue that I had with the tenosrflow-model-analysis installtion I am usually working with earlier version of pip to allow working with apache beam dataflow pipelines. !pip install pip==9.0.3 and for tensorflow-model-analysis (that is based on apache beam) it didnt work. I hope that I will not have more issues once i have it ready for running. So I am left with the widgets extension. I have never worked with them in the past, so it might be very trivial quesion.

Thanks, eilalan

[OrielResearchCure]

Hello all,

I am trying to have tensorflow-model-analysis working on datalab with no much success:

I am using python 2 My steps are the followings:

Step1:

!pip install --upgrade tensorflow
!pip install tensorflow_hub
!pip install --ignore-installed tensorflow-model-analysis

without using --ignore-installed, I get versions conflicts errors

Step 2: Reset kernel This allows me to import the latest tensorflow (version 1.12.0)

Step3:

import tensorflow as tf
import tensorflow_hub as hub
import logging
import csv
import pandas as pd
import os
from urllib import request
import zipfile
import numpy as np
import matplotlib.pyplot as plt
from tensorflow.python import debug as tf_debug
import random

Everything pass successfully and I was able to build a model and serve it.

Step 4: !jupyter nbextension install --py --symlink tensorflow_model_analysis # pass import tensorflow_model_analysis as tfma # fail

Error:

ImportErrorTraceback (most recent call last)
<ipython-input-48-f85e4d8fbd99> in <module>()
----> 1 import tensorflow_model_analysis as tfma

/usr/local/envs/py2env/lib/python2.7/site-packages/tensorflow_model_analysis/__init__.py in <module>()
     15 
     16 
---> 17 from tensorflow_model_analysis import view
     18 # pylint: disable=unused-import
     19 from tensorflow_model_analysis.api import tfma_unit as test

/usr/local/envs/py2env/lib/python2.7/site-packages/tensorflow_model_analysis/view/__init__.py in <module>()
     13 # limitations under the License.
     14 """Initializes TFMA's view rendering api."""
---> 15 from tensorflow_model_analysis.view.widget_view import render_plot
     16 from tensorflow_model_analysis.view.widget_view import render_slicing_metrics
     17 from tensorflow_model_analysis.view.widget_view import render_time_series

/usr/local/envs/py2env/lib/python2.7/site-packages/tensorflow_model_analysis/view/widget_view.py in <module>()
     18 from __future__ import print_function
     19 
---> 20 from tensorflow_model_analysis import constants
     21 from tensorflow_model_analysis.api import model_eval_lib
     22 import tensorflow_model_analysis.notebook.visualization as visualization

ImportError: cannot import name constants

I tried to install the other required extensions, with no much success:

  jupyter nbextension enable --py widgetsnbextension
  jupyter nbextension install --py --symlink tensorflow_model_analysis
  jupyter nbextension enable --py tensorflow_model_analysis

Please let me know what I am I doing wrong? How can I fix the environment

Many thanks for any input / reference, eilalan

[ojarjur]

@OrielResearchCure Look for errors in the output of the pip install ... commands.

Some of our packages are installed via Conda, and some via pip (since they are not available with Conda).

When a pip package has a dependency that was installed via Conda, and that dependency needs to be upgraded, the pip install will fail.

The way to work around that is to manually update the dependency via Conda.

I just tried this with tensorflow-model-analysis, and it needs to update pytz, which was installed via Conda.

I was able to get the import tensorflow_model_analysis line to work after running the following in a code cell, and then restarting my notebook kernel:

%%bash

source activate py2env
conda install -y pytz==2018.4 dill==0.2.8.2
pip install tensorflow==1.11.0 tensorflow-model-analysis==0.11.0

[OrielResearchCure]

Thank you.

I have tried the conda installation and received the below error. Please let me know what will be the easiest way to resolve this (I can try to install the libraries there, not sure what to do with google-big-query library) where you able to execute these commands (required for visualization. if needed, I can do it with separate jupyter notbook - https://github.com/tensorflow/model-analysis)

jupyter nbextension enable --py widgetsnbextension jupyter nbextension install --py --symlink tensorflow_model_analysis jupyter nbextension enable --py tensorflow_model_analysis

==> WARNING: A newer version of conda exists. <==
  current version: 4.5.10
  latest version: 4.5.11

Please update conda by running

    $ conda update -n base -c defaults conda

sqlite-3.25.3        | 1.9 MB    | ########## | 100% 
cryptography-2.4.1   | 606 KB    | ########## | 100% 
certifi-2018.10.15   | 139 KB    | ########## | 100% 
pytz-2018.4          | 211 KB    | ########## | 100% 
sip-4.18             | 264 KB    | ########## | 100% 
pyqt-4.11.4          | 3.3 MB    | ########## | 100% 
tk-8.6.8             | 3.1 MB    | ########## | 100% 
dill-0.2.8.2         | 110 KB    | ########## | 100% 
qt-4.8.7             | 34.1 MB   | ########## | 100% 
python-2.7.15        | 12.8 MB   | ########## | 100% 
openssl-1.1.1a       | 5.0 MB    | ########## | 100% 
twisted 18.7.0 requires PyHamcrest>=1.9.0, which is not installed.
pandas-gbq 0.3.0 has requirement google-cloud-bigquery>=0.28.0, but you'll have google-cloud-bigquery 0.25.0 which is incompatible.
google-cloud-monitoring 0.28.0 has requirement google-cloud-core<0.29dev,>=0.28.0, but you'll have google-cloud-core 0.25.0 which is incompatible.
google-cloud-dataflow 2.0.0 has requirement dill==0.2.6, but you'll have dill 0.2.8.2 which is incompatible.
google-cloud-dataflow 2.0.0 has requirement google-apitools==0.5.10, but you'll have google-apitools 0.5.20 which is incompatible.
google-cloud-dataflow 2.0.0 has requirement google-cloud-bigquery<0.24.0,>=0.23.0, but you'll have google-cloud-bigquery 0.25.0 which is incompatible.
google-cloud-dataflow 2.0.0 has requirement httplib2<0.10,>=0.8, but you'll have httplib2 0.11.3 which is incompatible.
google-cloud-dataflow 2.0.0 has requirement protobuf==3.2.0, but you'll have protobuf 3.6.1 which is incompatible.

Many thanks, eilalan

[OrielResearchCure]

Sorry. forgot to restart the kernel. The import is working! thank you!!!!

Working on completing the visualization - error was fired at the second jupyter command. see below.

!jupyter nbextension enable --py widgetsnbextension --user
!jupyter nbextension install --py --symlink tensorflow_model_analysis --user
!jupyter nbextension enable --py tensorflow_model_analysis --user

returned error at the symlink:

Enabling notebook extension jupyter-js-widgets/extension...
      - Validating: OK
/usr/local/envs/py2env/lib/python2.7/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters
/usr/local/envs/py2env/lib/python2.7/site-packages/scipy/optimize/_minimize.py:32: ImportWarning: Not importing directory '/usr/local/envs/py2env/lib/python2.7/site-packages/scipy/optimize/lbfgsb': missing __init__.py
  from .lbfgsb import _minimize_lbfgsb
/usr/local/envs/py2env/lib/python2.7/site-packages/scipy/spatial/__init__.py:95: ImportWarning: Not importing directory '/usr/local/envs/py2env/lib/python2.7/site-packages/scipy/spatial/qhull': missing __init__.py
  from .qhull import *
Installing /usr/local/envs/py2env/lib/python2.7/site-packages/tensorflow_model_analysis/static -> tfma_widget_js
Symlinking: /content/.local/share/jupyter/nbextensions/tfma_widget_js -> /usr/local/envs/py2env/lib/python2.7/site-packages/tensorflow_model_analysis/static
- Validating: problems found:
   X  require: /content/.local/share/jupyter/nbextensions/tfma_widget_js/extension.js
  OK section: notebook
Full spec: {'dest': 'tfma_widget_js', 'src': 'static', 'section': 'notebook', 'require': 'tfma_widget_js/extension'}

    To initialize this nbextension in the browser every time the notebook (or other app) loads:

          jupyter nbextension enable tensorflow_model_analysis --user --py

/usr/local/envs/py2env/lib/python2.7/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters
/usr/local/envs/py2env/lib/python2.7/site-packages/scipy/optimize/_minimize.py:32: ImportWarning: Not importing directory '/usr/local/envs/py2env/lib/python2.7/site-packages/scipy/optimize/lbfgsb': missing __init__.py
  from .lbfgsb import _minimize_lbfgsb
/usr/local/envs/py2env/lib/python2.7/site-packages/scipy/spatial/__init__.py:95: ImportWarning: Not importing directory '/usr/local/envs/py2env/lib/python2.7/site-packages/scipy/spatial/qhull': missing __init__.py
  from .qhull import *
Enabling notebook extension tfma_widget_js/extension...
      - Validating: OK

Please let me know if you have any idea. Thanks, eilalan

[OrielResearchCure]

Hello, trying to complete the Tensorflow-model-analysis prerequisite.

is this command: !jupyter nbextension install --py --symlink --debug tensorflow_model_analysis the same as this command: !jupyter nbextension install --py --symlink --debug tensorflow_model_analysis

running this: !jupyter nbextension list after the first command execution yields the following output:

Known nbextensions:
  config dir: /content/.jupyter/nbconfig
    notebook section
      jupyter-js-widgets/extension  enabled 
      - Validating: OK
      tfma_widget_js/extension  enabled 
      - Validating: OK
  config dir: /usr/local/envs/py2env/etc/jupyter/nbconfig
    notebook section
      jupyter-js-widgets/extension  enabled 
      - Validating: OK

Thanks, eilalan