mikael-alafriz-deel / lucid-sonic-dreams

MIT License
771 stars 157 forks source link

Tensorflow error once Hallucinating #2

Open Twangist opened 3 years ago

Twangist commented 3 years ago

I finally shook out installation problems (tensorflow==1.15 can be onerous, and in main.py line 32 the package dnnlib couldn't be found, but I worked around it). I ran your example 3-line script, first on my mp3 then on your "chemical_love.mp3". In both cases lucidsonicdreams blows up after 0% done hallucinating.

Output and stack trace, with my explicit home dir replaced by '~':


Preparing style... Preparing audio... Loading effects...

Doing math...

Hallucinating...

0%| | 0/3876 [00:00<?, ?it/s]2021-03-13 23:20:40.020396: E tensorflow/core/common_runtime/executor.cc:642] Executor failed to create kernel. Invalid argument: Conv2DCustomBackpropInputOp only supports NHWC. [[{{node G_synthesis_1/_Run/G_synthesis/cond/cond/cond/cond/128x128/Conv0_up/conv2d_transpose}}]] 0%| | 0/3876 [00:03<?, ?it/s] Traceback (most recent call last): File "~/_VENVs/Py3.7/lib/python3.7/site-packages/tensorflow-1.15.0-py3.7-macosx-10.9-x86_64.egg/tensorflow_core/python/client/session.py", line 1365, in _do_call return fn(*args) File "~/_VENVs/Py3.7/lib/python3.7/site-packages/tensorflow-1.15.0-py3.7-macosx-10.9-x86_64.egg/tensorflow_core/python/client/session.py", line 1350, in _run_fn target_list, run_metadata) File "~/_VENVs/Py3.7/lib/python3.7/site-packages/tensorflow-1.15.0-py3.7-macosx-10.9-x86_64.egg/tensorflow_core/python/client/session.py", line 1443, in _call_tf_sessionrun run_metadata) tensorflow.python.framework.errors_impl.InvalidArgumentError: Conv2DCustomBackpropInputOp only supports NHWC. [[{{node G_synthesis_1/_Run/G_synthesis/cond/cond/cond/cond/128x128/Conv0_up/conv2d_transpose}}]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "~/PycharmProjects/lucid-sonic-dreams/audiofile_to_mp4.py", line 19, in file_name = '~/PycharmProjects/lucid-sonic-dreams/chemical_love.mp4') File "~/_VENVs/Py3.7/lib/python3.7/site-packages/lucidsonicdreams-0.4-py3.7.egg/lucidsonicdreams/main.py", line 680, in hallucinate File "~/_VENVs/Py3.7/lib/python3.7/site-packages/lucidsonicdreams-0.4-py3.7.egg/lucidsonicdreams/main.py", line 528, in generate_frames File "~/PycharmProjects/lucid-sonic-dreams/dnnlib/tflib/network.py", line 629, in run mb_out = tf.get_default_session().run(out_expr, dict(zip(in_expr, mb_in))) File "~/_VENVs/Py3.7/lib/python3.7/site-packages/tensorflow-1.15.0-py3.7-macosx-10.9-x86_64.egg/tensorflow_core/python/client/session.py", line 956, in run run_metadata_ptr) File "~/_VENVs/Py3.7/lib/python3.7/site-packages/tensorflow-1.15.0-py3.7-macosx-10.9-x86_64.egg/tensorflow_core/python/client/session.py", line 1180, in _run feed_dict_tensor, options, run_metadata) File "~/_VENVs/Py3.7/lib/python3.7/site-packages/tensorflow-1.15.0-py3.7-macosx-10.9-x86_64.egg/tensorflow_core/python/client/session.py", line 1359, in _do_run run_metadata) File "~/_VENVs/Py3.7/lib/python3.7/site-packages/tensorflow-1.15.0-py3.7-macosx-10.9-x86_64.egg/tensorflow_core/python/client/session.py", line 1384, in _do_call raise type(e)(node_def, op, message) tensorflow.python.framework.errors_impl.InvalidArgumentError: Conv2DCustomBackpropInputOp only supports NHWC. [[node G_synthesis_1/_Run/G_synthesis/cond/cond/cond/cond/128x128/Conv0_up/conv2d_transpose (defined at ~/_VENVs/Py3.7/lib/python3.7/site-packages/tensorflow-1.15.0-py3.7-macosx-10.9-x86_64.egg/tensorflow_core/python/framework/ops.py:1748) ]]

Original stack trace for 'G_synthesis_1/_Run/G_synthesis/cond/cond/cond/cond/128x128/Conv0_up/conv2d_transpose': File "~/PycharmProjects/lucid-sonic-dreams/audiofile_to_mp4.py", line 19, in file_name = '~/PycharmProjects/lucid-sonic-dreams/chemical_love.mp4') File "~/_VENVs/Py3.7/lib/python3.7/site-packages/lucidsonicdreams-0.4-py3.7.egg/lucidsonicdreams/main.py", line 680, in hallucinate self.generate_frames() File "~/_VENVs/Py3.7/lib/python3.7/site-packages/lucidsonicdreams-0.4-py3.7.egg/lucidsonicdreams/main.py", line 528, in generate_frames .synthesis.run(w_batch, Gs_syn_kwargs) File "~/PycharmProjects/lucid-sonic-dreams/dnnlib/tflib/network.py", line 604, in run out_gpu = net_gpu.get_output_for(*in_gpu, return_as_list=True, *dynamic_kwargs) File "~/PycharmProjects/lucid-sonic-dreams/dnnlib/tflib/network.py", line 369, in get_output_for out_expr = self._build_func(final_inputs, build_kwargs) File "", line 556, in G_synthesis File "", line 555, in grow File "", line 549, in File "~/_VENVs/Py3.7/lib/python3.7/site-packages/tensorflow-1.15.0-py3.7-macosx-10.9-x86_64.egg/tensorflow_core/python/util/deprecation.py", line 507, in new_func return func(*args, kwargs) File "~/_VENVs/Py3.7/lib/python3.7/site-packages/tensorflow-1.15.0-py3.7-macosx-10.9-x86_64.egg/tensorflow_core/python/ops/control_flow_ops.py", line 1224, in cond orig_res_t, res_t = context_t.BuildCondBranch(true_fn) File "~/_VENVs/Py3.7/lib/python3.7/site-packages/tensorflow-1.15.0-py3.7-macosx-10.9-x86_64.egg/tensorflow_core/python/ops/control_flow_ops.py", line 1061, in BuildCondBranch original_result = fn() File "", line 554, in File "", line 555, in grow File "", line 549, in File "~/_VENVs/Py3.7/lib/python3.7/site-packages/tensorflow-1.15.0-py3.7-macosx-10.9-x86_64.egg/tensorflow_core/python/util/deprecation.py", line 507, in new_func return func(*args, *kwargs) File "~/_VENVs/Py3.7/lib/python3.7/site-packages/tensorflow-1.15.0-py3.7-macosx-10.9-x86_64.egg/tensorflow_core/python/ops/control_flow_ops.py", line 1224, in cond orig_res_t, res_t = context_t.BuildCondBranch(true_fn) File "~/_VENVs/Py3.7/lib/python3.7/site-packages/tensorflow-1.15.0-py3.7-macosx-10.9-x86_64.egg/tensorflow_core/python/ops/control_flow_ops.py", line 1061, in BuildCondBranch original_result = fn() File "", line 554, in File "", line 555, in grow File "", line 549, in File "~/_VENVs/Py3.7/lib/python3.7/site-packages/tensorflow-1.15.0-py3.7-macosx-10.9-x86_64.egg/tensorflow_core/python/util/deprecation.py", line 507, in new_func return func(args, kwargs) File "~/_VENVs/Py3.7/lib/python3.7/site-packages/tensorflow-1.15.0-py3.7-macosx-10.9-x86_64.egg/tensorflow_core/python/ops/control_flow_ops.py", line 1224, in cond orig_res_t, res_t = context_t.BuildCondBranch(true_fn) File "~/_VENVs/Py3.7/lib/python3.7/site-packages/tensorflow-1.15.0-py3.7-macosx-10.9-x86_64.egg/tensorflow_core/python/ops/control_flow_ops.py", line 1061, in BuildCondBranch original_result = fn() File "", line 554, in File "", line 555, in grow File "", line 549, in File "~/_VENVs/Py3.7/lib/python3.7/site-packages/tensorflow-1.15.0-py3.7-macosx-10.9-x86_64.egg/tensorflow_core/python/util/deprecation.py", line 507, in new_func return func(*args, *kwargs) File "~/_VENVs/Py3.7/lib/python3.7/site-packages/tensorflow-1.15.0-py3.7-macosx-10.9-x86_64.egg/tensorflow_core/python/ops/control_flow_ops.py", line 1224, in cond orig_res_t, res_t = context_t.BuildCondBranch(true_fn) File "~/_VENVs/Py3.7/lib/python3.7/site-packages/tensorflow-1.15.0-py3.7-macosx-10.9-x86_64.egg/tensorflow_core/python/ops/control_flow_ops.py", line 1061, in BuildCondBranch original_result = fn() File "", line 554, in File "", line 551, in grow File "", line 520, in block File "", line 191, in upscale2d_conv2d File "~/_VENVs/Py3.7/lib/python3.7/site-packages/tensorflow-1.15.0-py3.7-macosx-10.9-x86_64.egg/tensorflow_core/python/ops/nn_ops.py", line 2204, in conv2d_transpose name=name) File "~/_VENVs/Py3.7/lib/python3.7/site-packages/tensorflow-1.15.0-py3.7-macosx-10.9-x86_64.egg/tensorflow_core/python/ops/nn_ops.py", line 2275, in conv2d_transpose_v2 name=name) File "~/_VENVs/Py3.7/lib/python3.7/site-packages/tensorflow-1.15.0-py3.7-macosx-10.9-x86_64.egg/tensorflow_core/python/ops/gen_nn_ops.py", line 1407, in conv2d_backprop_input name=name) File "~/_VENVs/Py3.7/lib/python3.7/site-packages/tensorflow-1.15.0-py3.7-macosx-10.9-x86_64.egg/tensorflow_core/python/framework/op_def_library.py", line 794, in _apply_op_helper op_def=op_def) File "~/_VENVs/Py3.7/lib/python3.7/site-packages/tensorflow-1.15.0-py3.7-macosx-10.9-x86_64.egg/tensorflow_core/python/util/deprecation.py", line 507, in new_func return func(args, **kwargs) File "~/_VENVs/Py3.7/lib/python3.7/site-packages/tensorflow-1.15.0-py3.7-macosx-10.9-x86_64.egg/tensorflow_core/python/framework/ops.py", line 3357, in create_op attrs, op_def, compute_device) File "~/_VENVs/Py3.7/lib/python3.7/site-packages/tensorflow-1.15.0-py3.7-macosx-10.9-x86_64.egg/tensorflow_core/python/framework/ops.py", line 3426, in _create_op_internal op_def=op_def) File "~/_VENVs/Py3.7/lib/python3.7/site-packages/tensorflow-1.15.0-py3.7-macosx-10.9-x86_64.egg/tensorflow_core/python/framework/ops.py", line 1748, in init self._traceback = tf_stack.extract_stack()

ss32 commented 3 years ago

Exact same error for me as well. It crashes out on the first frame

ss32 commented 3 years ago

This is likely related to TF 1.15 expecting a certain version of Cuda

edit: I have Cuda 11

2021-03-14 20:05:58.687912: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcudart.so.10.0'; dlerror: libcudart.so.10.0: cannot open shared object file: No such file or directory
2021-03-14 20:05:58.687970: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcublas.so.10.0'; dlerror: libcublas.so.10.0: cannot open shared object file: No such file or directory
2021-03-14 20:05:58.688010: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcufft.so.10.0'; dlerror: libcufft.so.10.0: cannot open shared object file: No such file or directory
2021-03-14 20:05:58.688051: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcurand.so.10.0'; dlerror: libcurand.so.10.0: cannot open shared object file: No such file or directory
2021-03-14 20:05:58.688091: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcusolver.so.10.0'; dlerror: libcusolver.so.10.0: cannot open shared object file: No such file or directory
2021-03-14 20:05:58.688132: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcusparse.so.10.0'; dlerror: libcusparse.so.10.0: cannot open shared object file: No such file or directory
2021-03-14 20:05:58.688172: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcudnn.so.7'; dlerror: libcudnn.so.7: cannot open shared object file: No such file or directory
2021-03-14 20:05:58.688179: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1641] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
uhapppy commented 3 years ago

for me the solution was to install CUDA 10.0 instead of cuda 11 tensorflow 1.15 seem to work with cuda 10.0

fusiformfacearea commented 3 years ago

having this error too (and there is another thread about it)

orcharpark commented 3 years ago

having same problem, cuda 10 installed tho, any solutions?