I try to set up on the cloud server and use ngrok for port mapping.
docker logs -f mt3:
2023-04-21 00:59:58.119158: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-04-21 00:59:58.983517: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/nvidia/lib:/usr/local/nvidia/lib64
2023-04-21 00:59:58.983619: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/nvidia/lib:/usr/local/nvidia/lib64
2023-04-21 00:59:58.983638: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
/usr/local/lib/python3.8/dist-packages/flax/struct.py:136: FutureWarning: jax.tree_util.register_keypaths is deprecated, and will be removed in a future release. Please use `register_pytree_with_keys()` instead.
jax.tree_util.register_keypaths(data_clz, keypaths)
/usr/local/lib/python3.8/dist-packages/flax/struct.py:136: FutureWarning: jax.tree_util.register_keypaths is deprecated, and will be removed in a future release. Please use `register_pytree_with_keys()` instead.
jax.tree_util.register_keypaths(data_clz, keypaths)
/usr/local/lib/python3.8/dist-packages/flax/struct.py:136: FutureWarning: jax.tree_util.register_keypaths is deprecated, and will be removed in a future release. Please use `register_pytree_with_keys()` instead.
jax.tree_util.register_keypaths(data_clz, keypaths)
/usr/local/lib/python3.8/dist-packages/gradio/inputs.py:319: UserWarning: Usage of gradio.inputs is deprecated, and will not be supported in the future, please import your components from gradio.components
warnings.warn(
/usr/local/lib/python3.8/dist-packages/gradio/deprecation.py:40: UserWarning: `optional` parameter is deprecated, and it has no effect
warnings.warn(value)
/usr/local/lib/python3.8/dist-packages/gradio/outputs.py:98: UserWarning: Usage of gradio.outputs is deprecated, and will not be supported in the future, please import your components from gradio.components
warnings.warn(
/usr/local/lib/python3.8/dist-packages/gradio/deprecation.py:40: UserWarning: `allow_screenshot` parameter is deprecated, and it has no effect
warnings.warn(value)
/usr/local/lib/python3.8/dist-packages/gradio/deprecation.py:40: UserWarning: `enable_queue` is deprecated in `Interface()`, please use it within `launch()` instead.
warnings.warn(value)
/usr/local/lib/python3.8/dist-packages/gradio/interface.py:330: UserWarning: The `allow_flagging` parameter in `Interface` nowtakes a string value ('auto', 'manual', or 'never'), not a boolean. Setting parameter to: 'never'.
warnings.warn(
WARNING:jax._src.xla_bridge:No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)
After submitting a song:
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 890, in _bootstrap
self._bootstrap_inner()
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/local/lib/python3.8/dist-packages/anyio/_backends/_asyncio.py", line 867, in run
result = context.run(func, *args)
File "app.py", line 12, in inference
inference_model = InferenceModel('/home/user/app/checkpoints/' + str(model) + '/', str(model))
File "/home/user/app/imports_and_definitions.py", line 90, in __init__
self.restore_from_checkpoint(checkpoint_path)
File "/home/user/app/imports_and_definitions.py", line 125, in restore_from_checkpoint
train_state_initializer = t5x.utils.TrainStateInitializer(
File "/home/user/app/t5x/utils.py", line 1141, in __init__
self.global_train_state_shape = jax.eval_shape(
File "/usr/local/lib/python3.8/dist-packages/jax/_src/traceback_util.py", line 166, in reraise_with_filtered_traceback
return fun(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/jax/_src/api.py", line 2829, in eval_shape
out = pe.abstract_eval_fun(wrapped_fun.call_wrapped,
File "/usr/local/lib/python3.8/dist-packages/jax/_src/interpreters/partial_eval.py", line 664, in abstract_eval_fun
_, avals_out, _ = trace_to_jaxpr_dynamic(
File "/usr/local/lib/python3.8/dist-packages/jax/_src/profiler.py", line 314, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/jax/_src/interpreters/partial_eval.py", line 2049, in trace_to_jaxpr_dynamic
jaxpr, out_avals, consts = trace_to_subjaxpr_dynamic(
File "/usr/local/lib/python3.8/dist-packages/jax/_src/interpreters/partial_eval.py", line 2066, in trace_to_subjaxpr_dynamic
ans = fun.call_wrapped(*in_tracers_)
File "/usr/local/lib/python3.8/dist-packages/jax/_src/linear_util.py", line 166, in call_wrapped
ans = self.f(*args, **dict(self.params, **kwargs))
File "/usr/local/lib/python3.8/dist-packages/jax/_src/linear_util.py", line 166, in call_wrapped
ans = self.f(*args, **dict(self.params, **kwargs))
File "/home/user/app/t5x/utils.py", line 1135, in initialize_train_state
return train_state_lib.FlaxOptimTrainState.create(
File "/home/user/app/t5x/train_state.py", line 132, in create
other_variables, params = flax.core.pop(model_variables, 'params')
jax._src.traceback_util.UnfilteredStackTrace: AttributeError: module 'flax.core' has no attribute 'pop'
The stack trace below excludes JAX-internal frames.
The preceding is the original exception that occurred, unmodified.
--------------------
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/gradio/routes.py", line 401, in run_predict
output = await app.get_blocks().process_api(
File "/usr/local/lib/python3.8/dist-packages/gradio/blocks.py", line 1302, in process_api
result = await self.call_function(
File "/usr/local/lib/python3.8/dist-packages/gradio/blocks.py", line 1025, in call_function
prediction = await anyio.to_thread.run_sync(
File "/usr/local/lib/python3.8/dist-packages/anyio/to_thread.py", line 31, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "/usr/local/lib/python3.8/dist-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
return await future
File "/usr/lib/python3.8/asyncio/futures.py", line 260, in __await__
yield self # This tells Task to wait for completion.
File "/usr/lib/python3.8/asyncio/tasks.py", line 349, in __wakeup
future.result()
File "/usr/lib/python3.8/asyncio/futures.py", line 178, in result
raise self._exception
File "/usr/local/lib/python3.8/dist-packages/anyio/_backends/_asyncio.py", line 867, in run
result = context.run(func, *args)
File "app.py", line 12, in inference
inference_model = InferenceModel('/home/user/app/checkpoints/' + str(model) + '/', str(model))
File "/home/user/app/imports_and_definitions.py", line 90, in __init__
self.restore_from_checkpoint(checkpoint_path)
File "/home/user/app/imports_and_definitions.py", line 125, in restore_from_checkpoint
train_state_initializer = t5x.utils.TrainStateInitializer(
File "/home/user/app/t5x/utils.py", line 1141, in __init__
self.global_train_state_shape = jax.eval_shape(
File "/home/user/app/t5x/utils.py", line 1135, in initialize_train_state
return train_state_lib.FlaxOptimTrainState.create(
File "/home/user/app/t5x/train_state.py", line 132, in create
other_variables, params = flax.core.pop(model_variables, 'params')
AttributeError: module 'flax.core' has no attribute 'pop'
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 890, in _bootstrap
self._bootstrap_inner()
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/local/lib/python3.8/dist-packages/anyio/_backends/_asyncio.py", line 867, in run
result = context.run(func, *args)
File "app.py", line 12, in inference
inference_model = InferenceModel('/home/user/app/checkpoints/' + str(model) + '/', str(model))
File "/home/user/app/imports_and_definitions.py", line 90, in __init__
self.restore_from_checkpoint(checkpoint_path)
File "/home/user/app/imports_and_definitions.py", line 125, in restore_from_checkpoint
train_state_initializer = t5x.utils.TrainStateInitializer(
File "/home/user/app/t5x/utils.py", line 1141, in __init__
self.global_train_state_shape = jax.eval_shape(
File "/usr/local/lib/python3.8/dist-packages/jax/_src/traceback_util.py", line 166, in reraise_with_filtered_traceback
return fun(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/jax/_src/api.py", line 2829, in eval_shape
out = pe.abstract_eval_fun(wrapped_fun.call_wrapped,
File "/usr/local/lib/python3.8/dist-packages/jax/_src/interpreters/partial_eval.py", line 664, in abstract_eval_fun
_, avals_out, _ = trace_to_jaxpr_dynamic(
File "/usr/local/lib/python3.8/dist-packages/jax/_src/profiler.py", line 314, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/jax/_src/interpreters/partial_eval.py", line 2049, in trace_to_jaxpr_dynamic
jaxpr, out_avals, consts = trace_to_subjaxpr_dynamic(
File "/usr/local/lib/python3.8/dist-packages/jax/_src/interpreters/partial_eval.py", line 2066, in trace_to_subjaxpr_dynamic
ans = fun.call_wrapped(*in_tracers_)
File "/usr/local/lib/python3.8/dist-packages/jax/_src/linear_util.py", line 166, in call_wrapped
ans = self.f(*args, **dict(self.params, **kwargs))
File "/usr/local/lib/python3.8/dist-packages/jax/_src/linear_util.py", line 166, in call_wrapped
ans = self.f(*args, **dict(self.params, **kwargs))
File "/home/user/app/t5x/utils.py", line 1135, in initialize_train_state
return train_state_lib.FlaxOptimTrainState.create(
File "/home/user/app/t5x/train_state.py", line 132, in create
other_variables, params = flax.core.pop(model_variables, 'params')
jax._src.traceback_util.UnfilteredStackTrace: AttributeError: module 'flax.core' has no attribute 'pop'
The stack trace below excludes JAX-internal frames.
The preceding is the original exception that occurred, unmodified.
--------------------
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/gradio/routes.py", line 401, in run_predict
output = await app.get_blocks().process_api(
File "/usr/local/lib/python3.8/dist-packages/gradio/blocks.py", line 1302, in process_api
result = await self.call_function(
File "/usr/local/lib/python3.8/dist-packages/gradio/blocks.py", line 1025, in call_function
prediction = await anyio.to_thread.run_sync(
File "/usr/local/lib/python3.8/dist-packages/anyio/to_thread.py", line 31, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "/usr/local/lib/python3.8/dist-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
return await future
I try to set up on the cloud server and use ngrok for port mapping.
docker logs -f mt3:
After submitting a song: