TimoBolkart / voca

This codebase demonstrates how to synthesize realistic 3D character animations given an arbitrary speech signal and a static character mesh.
https://voca.is.tue.mpg.de/en
1.13k stars 274 forks source link

not working Still! #81

Open 007rohitSaini opened 2 years ago

007rohitSaini commented 2 years ago

python run_voca.py --tf_model_fname './model/gstep_52280.model' --ds_fname './ds_graph/output_graph.pb' --audio_fname './audio/test_sentence.wav' --template_fname './template/FLAME_sample.ply' --condition_idx 3 --out_path './animation_output' 2021-09-30 21:04:59.135478: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory 2021-09-30 21:04:59.135506: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.

Traceback (most recent call last): File "run_voca.py", line 59, in inference(tf_model_fname, ds_fname, audio_fname, template_fname, condition_idx, out_path, str2bool(args.visualize), uv_template_fname=uv_template_fname, texture_img_fname=texture_img_fname) File "/home/rohit_saini/Desktop/voca/utils/inference.py", line 109, in inference processed_audio = process_audio(ds_fname, audio, sample_rate) File "/home/rohit_saini/Desktop/voca/utils/inference.py", line 43, in process_audio return audio_handler.process(tmp_audio)['subj']['seq']['audio'] File "/home/rohit_saini/Desktop/voca/utils/audio_handler.py", line 53, in process return self.convert_to_deepspeech(audio) File "/home/rohit_saini/Desktop/voca/utils/audio_handler.py", line 102, in convert_to_deepspeech input_tensor = graph.get_tensor_by_name('deepspeech/input_node:0') File "/home/rohit_saini/Desktop/voca/.virtualenvs/voca/lib/python3.8/site-packages/tensorflow/python/framework/ops.py", line 3935, in get_tensor_by_name return self.as_graph_element(name, allow_tensor=True, allow_operation=False) File "/home/rohit_saini/Desktop/voca/.virtualenvs/voca/lib/python3.8/site-packages/tensorflow/python/framework/ops.py", line 3759, in as_graph_element return self._as_graph_element_locked(obj, allow_tensor, allow_operation) File "/home/rohit_saini/Desktop/voca/.virtualenvs/voca/lib/python3.8/site-packages/tensorflow/python/framework/ops.py", line 3799, in _as_graph_element_locked raise KeyError("The name %s refers to a Tensor which does not " KeyError: "The name 'deepspeech/input_node:0' refers to a Tensor which does not exist. The operation, 'deepspeech/input_node', does not exist in the graph."

TimoBolkart commented 2 years ago

I cannot reproduce this behavior. Can you add a breakpoint after this line in the audio handler and print the loaded tensorflow graph?

TimoBolkart commented 2 years ago

As it does not find the input node, you could for instance print the output of [placeholder for op in tf.get_default_graph().get_operations() if op.type=='Placeholder' for placeholder in op.values()] which should return the two input placeholders input_node and input_lengths?

007rohitSaini commented 2 years ago

can you please elaborate how to do it sir

007rohitSaini commented 2 years ago

i did this with tf.io.gfile.GFile(self.config['deepspeech_graph_fname'], "rb") as f: graph_def = tf.compat.v1.GraphDef() graph_def.ParseFromString(f.read())

    graph = tf.compat.v1.get_default_graph()
    tf.import_graph_def(graph_def, name="deepspeech")
    for op in tf.compat.v1.get_default_graph().get_operations():
        if op.type=='Placeholder':
            for placeholder in op.values():
                print(placeholder)
    input_tensor = graph.get_tensor_by_name('deepspeech/input_node:0')
    seq_length = graph.get_tensor_by_name('deepspeech/input_lengths:0')
    layer_6 = graph.get_tensor_by_name('deepspeech/logits:0')

and the result was

python run_voca.py --tf_model_fname './model/gstep_52280.model' --ds_fname './ds_graph/output_graph.pb' --audio_fname './audio/test_sentence.wav' --template_fname './template/FLAME_sample.ply' --condition_idx 3 --out_path './animation_output' 2021-10-26 14:48:53.509341: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory 2021-10-26 14:48:53.509368: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine. Tensor("input_node:0", shape=(None, None, 494), dtype=float32) Tensor("input_lengths:0", shape=(None,), dtype=int32) Traceback (most recent call last): File "run_voca.py", line 59, in inference(tf_model_fname, ds_fname, audio_fname, template_fname, condition_idx, out_path, str2bool(args.visualize), uv_template_fname=uv_template_fname, texture_img_fname=texture_img_fname) File "/home/rohit_saini/Desktop/voca/utils/inference.py", line 109, in inference processed_audio = process_audio(ds_fname, audio, sample_rate) File "/home/rohit_saini/Desktop/voca/utils/inference.py", line 43, in process_audio return audio_handler.process(tmp_audio)['subj']['seq']['audio'] File "/home/rohit_saini/Desktop/voca/utils/audio_handler.py", line 53, in process return self.convert_to_deepspeech(audio) File "/home/rohit_saini/Desktop/voca/utils/audio_handler.py", line 106, in convert_to_deepspeech input_tensor = graph.get_tensor_by_name('deepspeech/input_node:0') File "/home/rohit_saini/Desktop/voca/.virtualenvs/voca/lib/python3.8/site-packages/tensorflow/python/framework/ops.py", line 3935, in get_tensor_by_name return self.as_graph_element(name, allow_tensor=True, allow_operation=False) File "/home/rohit_saini/Desktop/voca/.virtualenvs/voca/lib/python3.8/site-packages/tensorflow/python/framework/ops.py", line 3759, in as_graph_element return self._as_graph_element_locked(obj, allow_tensor, allow_operation) File "/home/rohit_saini/Desktop/voca/.virtualenvs/voca/lib/python3.8/site-packages/tensorflow/python/framework/ops.py", line 3799, in _as_graph_element_locked raise KeyError("The name %s refers to a Tensor which does not " KeyError: "The name 'deepspeech/input_node:0' refers to a Tensor which does not exist. The operation, 'deepspeech/input_node', does not exist in the graph."

please look into the matter

007rohitSaini commented 2 years ago

also i printed graph and this was the result <tensorflow.python.framework.ops.Graph object at 0x7f2f02281dc0>

TimoBolkart commented 2 years ago

can you please elaborate how to do it sir

Please run print([placeholder for op in tf.get_default_graph().get_operations() if op.type=='Placeholder' for placeholder in op.values()]) after this line

TimoBolkart commented 2 years ago

Running your code it seems that the namespace "deepspeech" is missing. This might be due to some changes in your code, not sure. Does removing the deepspeech work for you, when you try to request input_tensor, seq_length, and layer_6 directly by the name of the tensors as follows:

input_tensor = graph.get_tensor_by_name('input_node:0')
seq_length = graph.get_tensor_by_name('input_lengths:0')
layer_6 = graph.get_tensor_by_name('logits:0')`
007rohitSaini commented 2 years ago

can you please elaborate how to do it sir

Please run print([placeholder for op in tf.get_default_graph().get_operations() if op.type=='Placeholder' for placeholder in op.values()]) after this line

the result was [<tf.Tensor 'input_node:0' shape=(None, None, 494) dtype=float32>, <tf.Tensor 'input_lengths:0' shape=(None,) dtype=int32>]

007rohitSaini commented 2 years ago

Running your code it seems that the namespace "deepspeech" is missing. This might be due to some changes in your code, not sure. Does removing the deepspeech work for you, when you try to request input_tensor, seq_length, and layer_6 directly by the name of the tensors as follows:

input_tensor = graph.get_tensor_by_name('input_node:0')
seq_length = graph.get_tensor_by_name('input_lengths:0')
layer_6 = graph.get_tensor_by_name('logits:0')`

raise KeyError("The name %s refers to a Tensor which does not " KeyError: "The name 'input_node:0' refers to a Tensor which does not exist. The operation, 'input_node', does not exist in the graph."

result

noulota13 commented 2 years ago

Running your code it seems that the namespace "deepspeech" is missing. This might be due to some changes in your code, not sure. Does removing the deepspeech work for you, when you try to request input_tensor, seq_length, and layer_6 directly by the name of the tensors as follows:

input_tensor = graph.get_tensor_by_name('input_node:0')
seq_length = graph.get_tensor_by_name('input_lengths:0')
layer_6 = graph.get_tensor_by_name('logits:0')`

raise KeyError("The name %s refers to a Tensor which does not " KeyError: "The name 'input_node:0' refers to a Tensor which does not exist. The operation, 'input_node', does not exist in the graph."

result

Did you find the solution ?

007rohitSaini commented 2 years ago

Running your code it seems that the namespace "deepspeech" is missing. This might be due to some changes in your code, not sure. Does removing the deepspeech work for you, when you try to request input_tensor, seq_length, and layer_6 directly by the name of the tensors as follows:

input_tensor = graph.get_tensor_by_name('input_node:0')
seq_length = graph.get_tensor_by_name('input_lengths:0')
layer_6 = graph.get_tensor_by_name('logits:0')`

raise KeyError("The name %s refers to a Tensor which does not " KeyError: "The name 'input_node:0' refers to a Tensor which does not exist. The operation, 'input_node', does not exist in the graph." result

Did you find the solution ?

no

xinleicol commented 1 year ago

just delete "deepspeech/*" at line 101-104, it works for me. input_tensor = graph.get_tensor_by_name('input_node:0') seq_length = graph.get_tensor_by_name('input_lengths:0') layer_6 = graph.get_tensor_by_name('logits:0')

xinleicol commented 1 year ago

If tensorflow is version 2.0 or higher, the code cannot run. Because gstep_52280.model is not compatible with versions above 2.0; although you can modify import tensorflow.compat.v1 as tf; tf.disable_v2_behavior() to use codes below 2.0, it still cannot solve the compatibility problem of the model; I try to use tensorflow2. 3. After running the code for an afternoon, I finally gave up. It is better to configure python3.6 version + tensorflow1.15 to try to run the code