Closed smithee77 closed 2 years ago
romi1502
commented
2 years ago Hi @smithee77,
I cannot reproduce your issue. The colab notebook works fine on my side and does not generate errors on !spleeter separate -o output/ audio_example.mp3.
You seem to have GPU related errors, so it may mean that you're running the colab on a different instance than I do (I use the default Backend Google Compute Engine Python 3).
smithee77
commented
2 years ago Hi @romi1502 thanks 4 your answer. I'm running it again, I also use default Backend Google Compute Engine Python 3, same as yours. I've tried with an standard Colab account and with a Pro account. In both I'm getting same error. I've also tried connecting via VPN using different countries (I don't know if this makes any difference)...and same error in the same point, using notebook as it, without changing anything :S
Really weird, will keep trying Regards and thanks again
smithee77
commented
2 years ago Solved. It has to be run in an NON-GPU session.
Description
Google colab notebook doesn't work. Thanks for you work and your support.
Step to reproduce
Just used https://colab.research.google.com/github/deezer/spleeter/blob/master/spleeter.ipynb in Google Colab Pro, GPU session I get an error in line: !spleeter separate -o output/ audio_example.mp3
Output
Traceback (most recent call last): File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1375, in _do_call return fn(*args) File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1360, in _run_fn target_list, run_metadata) File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1453, in _call_tf_sessionrun run_metadata) tensorflow.python.framework.errors_impl.UnknownError: 2 root error(s) found. (0) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above. [[{{node conv2d/Conv2D}}]] [[strided_slice_23/_907]] (1) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above. [[{{node conv2d/Conv2D}}]] 0 successful operations. 0 derived errors ignored.
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/usr/local/bin/spleeter", line 8, in
sys.exit(entrypoint())
File "/usr/local/lib/python3.7/dist-packages/spleeter/main.py", line 256, in entrypoint
spleeter()
File "/usr/local/lib/python3.7/dist-packages/typer/main.py", line 214, in call
return get_command(self)(*args, kwargs)
File "/usr/local/lib/python3.7/dist-packages/click/core.py", line 829, in call
return self.main(args, kwargs)
File "/usr/local/lib/python3.7/dist-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.7/dist-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.7/dist-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, ctx.params)
File "/usr/local/lib/python3.7/dist-packages/click/core.py", line 610, in invoke
return callback(args, kwargs)
File "/usr/local/lib/python3.7/dist-packages/typer/main.py", line 497, in wrapper
return callback(use_params) # type: ignore
File "/usr/local/lib/python3.7/dist-packages/spleeter/main.py", line 137, in separate
synchronous=False,
File "/usr/local/lib/python3.7/dist-packages/spleeter/separator.py", line 378, in separate_to_file
sources = self.separate(waveform, audio_descriptor)
File "/usr/local/lib/python3.7/dist-packages/spleeter/separator.py", line 319, in separate
return self._separate_tensorflow(waveform, audio_descriptor)
File "/usr/local/lib/python3.7/dist-packages/spleeter/separator.py", line 301, in _separate_tensorflow
prediction = next(prediction_generator)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_estimator/python/estimator/estimator.py", line 631, in predict
preds_evaluated = mon_sess.run(predictions)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/training/monitored_session.py", line 779, in run
run_metadata=run_metadata)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/training/monitored_session.py", line 1284, in run
run_metadata=run_metadata)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/training/monitored_session.py", line 1385, in run
raise six.reraise(original_exc_info)
File "/usr/local/lib/python3.7/dist-packages/six.py", line 703, in reraise
raise value
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/training/monitored_session.py", line 1370, in run
return self._sess.run(args, kwargs)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/training/monitored_session.py", line 1443, in run
run_metadata=run_metadata)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/training/monitored_session.py", line 1201, in run
return self._sess.run(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 968, in run
run_metadata_ptr)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1191, in _run
feed_dict_tensor, options, run_metadata)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1369, in _do_run
run_metadata)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1394, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.UnknownError: 2 root error(s) found.
(0) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
[[node conv2d/Conv2D (defined at /lib/python3.7/dist-packages/spleeter/model/functions/unet.py:109) ]]
[[strided_slice_23/_907]]
(1) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
[[node conv2d/Conv2D (defined at /lib/python3.7/dist-packages/spleeter/model/functions/unet.py:109) ]]
0 successful operations.
0 derived errors ignored.
Errors may have originated from an input operation. Input Source operations connected to node conv2d/Conv2D: strided_slice_3 (defined at /lib/python3.7/dist-packages/spleeter/model/init.py:307)
Input Source operations connected to node conv2d/Conv2D: strided_slice_3 (defined at /lib/python3.7/dist-packages/spleeter/model/init.py:307)
Original stack trace for 'conv2d/Conv2D': File "/bin/spleeter", line 8, in
sys.exit(entrypoint())
File "/lib/python3.7/dist-packages/spleeter/main.py", line 256, in entrypoint
spleeter()
File "/lib/python3.7/dist-packages/typer/main.py", line 214, in call
return get_command(self)(*args, kwargs)
File "/lib/python3.7/dist-packages/click/core.py", line 829, in call
return self.main(args, kwargs)
File "/lib/python3.7/dist-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/lib/python3.7/dist-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/lib/python3.7/dist-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, ctx.params)
File "/lib/python3.7/dist-packages/click/core.py", line 610, in invoke
return callback(args, kwargs)
File "/lib/python3.7/dist-packages/typer/main.py", line 497, in wrapper
return callback(use_params) # type: ignore
File "/lib/python3.7/dist-packages/spleeter/main.py", line 137, in separate
synchronous=False,
File "/lib/python3.7/dist-packages/spleeter/separator.py", line 378, in separate_to_file
sources = self.separate(waveform, audio_descriptor)
File "/lib/python3.7/dist-packages/spleeter/separator.py", line 319, in separate
return self._separate_tensorflow(waveform, audio_descriptor)
File "/lib/python3.7/dist-packages/spleeter/separator.py", line 301, in _separate_tensorflow
prediction = next(prediction_generator)
File "/lib/python3.7/dist-packages/tensorflow_estimator/python/estimator/estimator.py", line 613, in predict
self.config)
File "/lib/python3.7/dist-packages/tensorflow_estimator/python/estimator/estimator.py", line 1163, in _call_model_fn
model_fn_results = self._model_fn(features=features, kwargs)
File "/lib/python3.7/dist-packages/spleeter/model/init.py", line 568, in model_fn
return builder.build_predict_model()
File "/lib/python3.7/dist-packages/spleeter/model/init.py", line 516, in build_predict_model
tf.estimator.ModeKeys.PREDICT, predictions=self.outputs
File "/lib/python3.7/dist-packages/spleeter/model/init.py", line 318, in outputs
self._build_outputs()
File "/lib/python3.7/dist-packages/spleeter/model/init.py", line 499, in _build_outputs
self._outputs = self._build_output_waveform(self.masked_stfts)
File "/lib/python3.7/dist-packages/spleeter/model/init.py", line 342, in masked_stfts
self._build_masked_stfts()
File "/lib/python3.7/dist-packages/spleeter/model/init.py", line 465, in _build_masked_stfts
for instrument, mask in self.masks.items():
File "/lib/python3.7/dist-packages/spleeter/model/init.py", line 336, in masks
self._build_masks()
File "/lib/python3.7/dist-packages/spleeter/model/init.py", line 432, in _build_masks
output_dict = self.model_outputs
File "/lib/python3.7/dist-packages/spleeter/model/init.py", line 312, in model_outputs
self._build_model_outputs()
File "/lib/python3.7/dist-packages/spleeter/model/init.py", line 212, in _build_model_outputs
input_tensor, self._instruments, self._params["model"]["params"]
File "/lib/python3.7/dist-packages/spleeter/model/functions/unet.py", line 197, in unet
return apply(apply_unet, input_tensor, instruments, params)
File "/lib/python3.7/dist-packages/spleeter/model/functions/init.py", line 45, in apply
input_tensor, output_name=out_name, params=params or {}
File "/lib/python3.7/dist-packages/spleeter/model/functions/unet.py", line 109, in apply_unet
conv1 = conv2d_factory(conv_n_filters[0], (5, 5))(input_tensor)
File "/lib/python3.7/dist-packages/tensorflow/python/keras/engine/base_layer_v1.py", line 783, in call
outputs = call_fn(cast_inputs, *args, *kwargs)
File "/lib/python3.7/dist-packages/tensorflow/python/keras/layers/convolutional.py", line 249, in call
outputs = self._convolution_op(inputs, self.kernel)
File "/lib/python3.7/dist-packages/tensorflow/python/util/dispatch.py", line 206, in wrapper
return target(args, **kwargs)
File "/lib/python3.7/dist-packages/tensorflow/python/ops/nn_ops.py", line 1019, in convolution_v2
name=name)
File "/lib/python3.7/dist-packages/tensorflow/python/ops/nn_ops.py", line 1149, in convolution_internal
name=name)
File "/lib/python3.7/dist-packages/tensorflow/python/ops/nn_ops.py", line 2603, in _conv2d_expanded_batch
name=name)
File "/lib/python3.7/dist-packages/tensorflow/python/ops/gen_nn_ops.py", line 973, in conv2d
data_format=data_format, dilations=dilations, name=name)
File "/lib/python3.7/dist-packages/tensorflow/python/framework/op_def_library.py", line 750, in _apply_op_helper
attrs=attr_protos, op_def=op_def)
File "/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 3565, in _create_op_internal
op_def=op_def)
File "/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 2045, in init
self._traceback = tf_stack.extract_stack_for_node(self._c_op)