Tensorflow 2.2 support - Githubissues

IBM / tensorflow-large-model-support

Large Model Support in Tensorflow

Apache License 2.0

202 stars 38 forks source link

Tensorflow 2.2 support #42

Closed simulatio closed 4 years ago

simulatio commented 4 years ago

Is it possible to use LMS with Tensorflow 2.2 or is there a plan for this in the future?

jayfurmanek commented 4 years ago

Yes, but it needs some changes which we have done. We have it built internally and will release it soon in package form [1] and patch form [2]

[1] https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda-early-access/ [2] https://github.com/IBM/tensorflow-large-model-support

I'll keep this open and close it once its out

sahoo-17198 commented 4 years ago

How about tensorflow 2.1? On what version is it available? When I run this code:

import tensorflow as tf
tf.config.experimental.set_lms_enabled(True)

I get the error-

AttributeError: module 'tensorflow._api.v2.config.experimental' has no attribute 'set_lms_enabled'

Help. I have tried it on tf 2.1as well.

jayfurmanek commented 4 years ago

LMS is available for TF2.1 in two ways:

patch the TF2.1 code base with the patch in this repo and rebuild (https://github.com/IBM/tensorflow-large-model-support/#building-tensorflow-from-source-with-tensorflow-large-model-support)
use the WML CE version of TF2.1 located in this conda channel: https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda/#/

https://github.com/IBM/tensorflow-large-model-support/#installing-tensorflow-large-model-support

jayfurmanek commented 4 years ago

Hi again TF 2.2 with integrated LMS is available now in this channel:

https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda-early-access/

Please give it a try and let us know how it goes!

simulatio commented 4 years ago

Is the LMS enabled by default in this TF 2.2 version? I get the above attribute error when I try tf.config.experimental.set_lms_enabled(True)

smatzek commented 4 years ago

LMS should be available in the TensorFlow 2.1 and 2.2 versions in that channel. The enablement using tf.config.experimental.set_lms_enabled(True) should work.

It is possible that one or more TensorFlow packages have been installed into your environment from a different channel. Can you post the output of conda list | grep tensorflow ?

simulatio commented 4 years ago

Output is below

tensorflow                2.2.0           gpu_py37_970.g1bb2968    https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda-early-access
tensorflow-base           2.2.0           gpu_py37h8a81be8_0    defaults
tensorflow-estimator      2.2.0           py37_5bd33a6_1597.g1bb2968    https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda-early-access
tensorflow-gpu            2.2.0              970.g1bb2968    https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda-early-access

jayfurmanek commented 4 years ago

yeah - you have tensorflow-base from defaults. That's not a workable mix. Try conda install tensorflow-base=2.2.0=gpu_py37_2b96f36_162011.g4c42fc8

Hopefully that will get the right one on there.

simulatio commented 4 years ago

Thank you, needed to add --strict-channel-priority to conda command but got it working in the end.