search

IBM / MAX-Text-Sentiment-Classifier

Detect the sentiment captured in short pieces of text
https://developer.ibm.com/exchanges/models/all/max-text-sentiment-classifier/
Apache License 2.0
58 stars 31 forks source link

OpenShift 4.3.3 error #31

Open WaughB opened 4 years ago

WaughB commented 4 years ago

Tried to recreate your project in OpenShift 4.3.3 and got CrashLoopBackOff . These are the logs when I checked the logs via OpenShift Online:

WARNING:tensorflow:From /workspace/core/model.py:42: The name tf.logging.set_verbosity is deprecated. Please use tf.compat.v1.logging.set_verbosity instead.

WARNING:tensorflow:From /workspace/core/model.py:42: The name tf.logging.INFO is deprecated. Please use tf.compat.v1.logging.INFO instead.

WARNING:tensorflow:From /workspace/core/model.py:46: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

2020-03-13 16:33:22.240151: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-03-13 16:33:22.250517: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2300060000 Hz
2020-03-13 16:33:22.250944: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55f52f1f57e0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-03-13 16:33:22.250981: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
WARNING:tensorflow:From /workspace/core/model.py:47: load (from tensorflow.python.saved_model.loader_impl) is deprecated and will be removed in a future version.
Instructions for updating:
This function will only be available through the v1 compatibility library as tf.compat.v1.saved_model.loader.load or tf.compat.v1.saved_model.load. There will be a new function for importing SavedModels in Tensorflow 2.0.
INFO:tensorflow:Restoring parameters from assets/variables/variables
Killed

Is there something I need to configure differently to run this project in more recent versions of OpenShift? Or will the project have to be run only in version 3.11? The issues seem to stem around tensorflow, is there a way I can specify the version I need?

To recreate: Developer view -> Add -> Container Image -> Image Name = codait/max-text-sentiment-classifier -> default the rest of the values, also under "Advanced Options" left the "Create a route to the application" checked. -> Create.

I was able to recreate the project on my desktop using:

docker run -it -p 5000:5000 codait/max-text-sentiment-classifier

and it worked as expected. I still received tensorflow warnings (below), but it did not interfere with running the image. 

WARNING:tensorflow:From /workspace/core/model.py:42: The name tf.logging.set_verbosity is deprecated. Please use tf.compat.v1.logging.set_verbosity instead.

WARNING:tensorflow:From /workspace/core/model.py:42: The name tf.logging.INFO is deprecated. Please use tf.compat.v1.logging.INFO instead.

WARNING:tensorflow:From /workspace/core/model.py:46: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

2020-03-13 12:22:26.515979: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-03-13 12:22:26.530494: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2904010000 Hz
2020-03-13 12:22:26.530666: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55ff74a0f2b0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-03-13 12:22:26.530727: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
WARNING:tensorflow:From /workspace/core/model.py:47: load (from tensorflow.python.saved_model.loader_impl) is deprecated and will be removed in a future version.
Instructions for updating:
This function will only be available through the v1 compatibility library as tf.compat.v1.saved_model.loader.load or tf.compat.v1.saved_model.load. There will be a new function for importing SavedModels in Tensorflow 2.0.
INFO:tensorflow:Restoring parameters from assets/variables/variables
2020-03-13 12:22:27.359251: W tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 93763584 exceeds 10% of system memory.
WARNING:tensorflow:From /workspace/core/bert/tokenization.py:121: The name tf.gfile.GFile is deprecated. Please use tf.io.gfile.GFile instead.

Let me know if there is anymore information or tests I can perform to help.

ptitzler commented 4 years ago

The latest version we've tested is 3.x I'll try it with version 4.3.1, which is the latest version I have access to atm. I'm suspecting that you are running out of resources. The minimum configuration for this model is 4GB RAM.

ptitzler commented 4 years ago

Tested deployment on OpenShift 4.3.1, which is currently in Beta on IBM Cloud, without any issues. Are you using by any chance the starter plan, which is capped at 2GB RAM?

image

WaughB commented 4 years ago

I am using the starter plan for developing. I will report back if it works with the larger OpenShift instance. Thank you so much for responding!