Open xhluca opened 3 years ago
Seems like the issue still persists when run locally. The issue seems to stem from the version mismatch between tensorflow and the Keras-2.3.1. Unfortunately, I tried downgrading tensorflow to 1.x and change keras to tensorflow.keras, but the app still does not work correctly. I will keep it ignored until further investigation.
Seems like
dash-fashion-mnist-explorer
does not deploy on circleci. Will deployignore the app for now but we should investigate it later.Show circleci failure logs
``` ====> Deploying dash-fashion-mnist-explorer dashr: false dash python version: dash==1.7.0 exists: true remote-sha: ====> Initializing repository ====> Ensuring code is up to date Copying updated app source Python app detected, injecting common python-specific files ====> Deploying Counting objects: 34, done. Delta compression using up to 36 threads. Compressing objects: 100% (33/33), done. Writing objects: 100% (34/34), 32.66 MiB | 2.94 MiB/s, done. Total 34 (delta 0), reused 0 (delta 0) remote: -----> DDS disk space information remote: Filesystem Size Used Avail Use% Mounted on remote: overlay 2.0T 1.5T 512G 75% / remote: /dev/sdc 206G 117G 79G 60% /data remote: -----> DASH_APP_NAME => dash-fashion-mnist-explorer remote: -----> DASH_DOMAIN_BASE => dash-gallery.plotly.host remote: -----> DASH_PATH_ROUTING => 1 remote: -----> DASH_STREAMBED_DIRECT_IP => 172.17.0.1 remote: -----> DASH_LOGOUT_URL => /Manager/api/logout remote: -----> SCRIPT_NAME => /dash-fashion-mnist-explorer remote: -----> Cleaning up... remote: -----> Building dash-fashion-mnist-explorer from herokuish... remote: rehash: warning: skipping ca-certificates.pem,it does not contain exactly one certificate or CRL remote: 1 added, 0 removed; done. remote: Running hooks in /etc/ca-certificates/update.d... remote: remote: updates of cacerts keystore disabled. remote: done. remote: -----> Adding BUILD_ENV to build environment... remote: -----> Python app detected remote: -----> Installing pip remote: -----> Installing requirements with pip remote: Collecting setuptools>=41.0.0 (from tensorboard<1.15.0,>=1.14.0->tensorflow->-r /tmp/build/requirements.txt (line 5)) remote: Downloading https://files.pythonhosted.org/packages/9e/d4/b99a960314121a003e9f39c61dfde01a1010bb47661e193a7722f7f32d52/setuptools-54.2.0-py3-none-any.whl (785kB) remote: Installing collected packages: setuptools remote: Found existing installation: setuptools 39.0.1 remote: Uninstalling setuptools-39.0.1: remote: Successfully uninstalled setuptools-39.0.1 remote: Successfully installed setuptools-54.2.0 remote: -----> Discovering process types remote: Procfile declares types -> web remote: -----> Releasing dash-fashion-mnist-explorer (dokku/dash-fashion-mnist-explorer:latest)... remote: -----> Deploying dash-fashion-mnist-explorer (dokku/dash-fashion-mnist-explorer:latest)... remote: ! Predeploy command declared: 'python predeploy.py' remote: -----> App Procfile file found (/home/dokku/dash-fashion-mnist-explorer/DOKKU_PROCFILE) remote: DOKKU_SCALE declares scale -> web=1 remote: =====> Processing deployment checks remote: No CHECKS file found. Simple container checks will be performed. remote: For more efficient zero downtime deployments, create a CHECKS file. See http://dokku.viewdocs.io/dokku/deployment/zero-downtime-deploys/ for examples remote: -----> Attempting pre-flight checks (web.1) remote: Waiting for 10 seconds ... remote: 3be80fce7b898273adbf023137d00167fcf63f2e82ba15a108a0f760089f2db6 remote: App container failed to start!! remote: =====> dash-fashion-mnist-explorer web container output: remote: [2021-03-31 21:32:51 +0000] [12] [INFO] Starting gunicorn 19.9.0 remote: [2021-03-31 21:32:51 +0000] [12] [INFO] Listening at: http://0.0.0.0:5000 (12) remote: [2021-03-31 21:32:51 +0000] [12] [INFO] Using worker: sync remote: [2021-03-31 21:32:51 +0000] [189] [INFO] Booting worker with pid: 189 remote: Using TensorFlow backend. remote: /app/.heroku/python/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. remote: _np_qint8 = np.dtype([("qint8", np.int8, 1)]) remote: /app/.heroku/python/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. remote: _np_quint8 = np.dtype([("quint8", np.uint8, 1)]) remote: /app/.heroku/python/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. remote: _np_qint16 = np.dtype([("qint16", np.int16, 1)]) remote: /app/.heroku/python/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. remote: _np_quint16 = np.dtype([("quint16", np.uint16, 1)]) remote: /app/.heroku/python/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. remote: _np_qint32 = np.dtype([("qint32", np.int32, 1)]) remote: /app/.heroku/python/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. remote: np_resource = np.dtype([("resource", np.ubyte, 1)]) remote: /app/.heroku/python/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. remote: _np_qint8 = np.dtype([("qint8", np.int8, 1)]) remote: /app/.heroku/python/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. remote: _np_quint8 = np.dtype([("quint8", np.uint8, 1)]) remote: /app/.heroku/python/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. remote: _np_qint16 = np.dtype([("qint16", np.int16, 1)]) remote: /app/.heroku/python/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. remote: _np_quint16 = np.dtype([("quint16", np.uint16, 1)]) remote: /app/.heroku/python/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. remote: _np_qint32 = np.dtype([("qint32", np.int32, 1)]) remote: /app/.heroku/python/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. remote: np_resource = np.dtype([("resource", np.ubyte, 1)]) remote: [2021-03-31 21:33:01 +0000] [189] [ERROR] Exception in worker process remote: Traceback (most recent call last): remote: File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker remote: worker.init_process() remote: File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process remote: self.load_wsgi() remote: File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi remote: self.wsgi = self.app.wsgi() remote: File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi remote: self.callable = self.load() remote: File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load remote: return self.load_wsgiapp() remote: File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp remote: return util.import_app(self.app_uri) remote: File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/util.py", line 350, in import_app remote: __import__(module) remote: File "/app/app.py", line 30, in
remote: model = load_model("trained_data/fashion_mnist_cnn.h5")
remote: File "/app/.heroku/python/lib/python3.6/site-packages/keras/engine/saving.py", line 492, in load_wrapper
remote: return load_function(*args, **kwargs)
remote: File "/app/.heroku/python/lib/python3.6/site-packages/keras/engine/saving.py", line 584, in load_model
remote: model = _deserialize_model(h5dict, custom_objects, compile)
remote: File "/app/.heroku/python/lib/python3.6/site-packages/keras/engine/saving.py", line 273, in _deserialize_model
remote: model_config = json.loads(model_config.decode('utf-8'))
remote: AttributeError: 'str' object has no attribute 'decode'
remote: [2021-03-31 21:33:01 +0000] [189] [INFO] Worker exiting (pid: 189)
remote: [2021-03-31 21:33:01 +0000] [12] [INFO] Shutting down: Master
remote: [2021-03-31 21:33:01 +0000] [12] [INFO] Reason: Worker failed to boot.
remote: [2021-03-31 21:33:04 +0000] [15] [INFO] Starting gunicorn 19.9.0
remote: [2021-03-31 21:33:04 +0000] [15] [INFO] Listening at: http://0.0.0.0:5000 (15)
remote: [2021-03-31 21:33:04 +0000] [15] [INFO] Using worker: sync
remote: [2021-03-31 21:33:04 +0000] [191] [INFO] Booting worker with pid: 191
remote: Using TensorFlow backend.
remote: /app/.heroku/python/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
remote: _np_qint8 = np.dtype([("qint8", np.int8, 1)])
remote: /app/.heroku/python/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
remote: _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
remote: /app/.heroku/python/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
remote: _np_qint16 = np.dtype([("qint16", np.int16, 1)])
remote: /app/.heroku/python/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
remote: _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
remote: /app/.heroku/python/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
remote: _np_qint32 = np.dtype([("qint32", np.int32, 1)])
remote: /app/.heroku/python/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
remote: np_resource = np.dtype([("resource", np.ubyte, 1)])
remote: [2021-03-31 21:33:05 +0000] [15] [INFO] Handling signal: term
remote: /app/.heroku/python/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
remote: _np_qint8 = np.dtype([("qint8", np.int8, 1)])
remote: /app/.heroku/python/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
remote: _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
remote: /app/.heroku/python/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
remote: _np_qint16 = np.dtype([("qint16", np.int16, 1)])
remote: /app/.heroku/python/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
remote: _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
remote: /app/.heroku/python/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
remote: _np_qint32 = np.dtype([("qint32", np.int32, 1)])
remote: /app/.heroku/python/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
remote: np_resource = np.dtype([("resource", np.ubyte, 1)])
remote: [2021-03-31 21:33:09 +0000] [191] [ERROR] Exception in worker process
remote: Traceback (most recent call last):
remote: File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
remote: worker.init_process()
remote: File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process
remote: self.load_wsgi()
remote: File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
remote: self.wsgi = self.app.wsgi()
remote: File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
remote: self.callable = self.load()
remote: File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
remote: return self.load_wsgiapp()
remote: File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
remote: return util.import_app(self.app_uri)
remote: File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/util.py", line 350, in import_app
remote: __import__(module)
remote: File "/app/app.py", line 30, in
remote: model = load_model("trained_data/fashion_mnist_cnn.h5")
remote: File "/app/.heroku/python/lib/python3.6/site-packages/keras/engine/saving.py", line 492, in load_wrapper
remote: return load_function(*args, **kwargs)
remote: File "/app/.heroku/python/lib/python3.6/site-packages/keras/engine/saving.py", line 584, in load_model
remote: model = _deserialize_model(h5dict, custom_objects, compile)
remote: File "/app/.heroku/python/lib/python3.6/site-packages/keras/engine/saving.py", line 273, in _deserialize_model
remote: model_config = json.loads(model_config.decode('utf-8'))
remote: AttributeError: 'str' object has no attribute 'decode'
remote: [2021-03-31 21:33:09 +0000] [191] [INFO] Worker exiting (pid: 191)
remote: Traceback (most recent call last):
remote: File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/arbiter.py", line 225, in run
remote: handler()
remote: File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/arbiter.py", line 260, in handle_term
remote: raise StopIteration
remote: StopIteration
remote: During handling of the above exception, another exception occurred:
remote: Traceback (most recent call last):
remote: File "/app/.heroku/python/bin/gunicorn", line 11, in
remote: sys.exit(run())
remote: File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 61, in run
remote: WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]").run()
remote: File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/app/base.py", line 223, in run
remote: super(Application, self).run()
remote: File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/app/base.py", line 72, in run
remote: Arbiter(self).run()
remote: File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/arbiter.py", line 228, in run
remote: self.halt()
remote: File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/arbiter.py", line 345, in halt
remote: self.stop()
remote: File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/arbiter.py", line 393, in stop
remote: time.sleep(0.1)
remote: File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/arbiter.py", line 245, in handle_chld
remote: self.reap_workers()
remote: File "/app/.heroku/python/lib/python3.6/site-packages/gunicorn/arbiter.py", line 525, in reap_workers
remote: raise HaltServer(reason, self.WORKER_BOOT_ERROR)
remote: gunicorn.errors.HaltServer:
remote: =====> end dash-fashion-mnist-explorer web container output
To ********************************/GIT/dash-fashion-mnist-explorer
! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to '********************************/GIT/dash-fashion-mnist-explorer'
Exited with code exit status 1
CircleCI received exit code 1
```