Closed hvgazula closed 1 month ago
While I managed to go past this step by modifying the build
and call
in SampleResolution
by replacing
if input_shape:
self.add_batchsize = True
self.min_res_tens = tf.convert_to_tensor(self.min_res, dtype='float32')
in build(...)
with
self.min_res_tens = tf.convert_to_tensor(self.min_res, dtype='float32')
if input_shape:
self.add_batchsize = True
self.min_res_tens = tf.expand_dims(self.min_res_tens, 0)
and subsequently replacing self.min_res_tens = tf.tile(tf.expand_dims(self.min_res_tens, 0), tile_shape)
in call(...)
with self.min_res_tens = tf.tile(self.min_res_tens, tile_shape)
, it now throws a new error that goes along the lines of
Traceback (most recent call last):
File "/net/vast-storage/scratch/vast/gablab/hgazula/SynthSeg/scripts/misc/synth_202408.py", line 291, in <module>
[image, labels] = lab_to_im_model.predict(model_inputs)
File "/om2/user/hgazula/venvs/ss2.15/lib/python3.9/site-packages/keras/src/utils/traceback_utils.py", line 70, in error_handler
raise e.with_traceback(filtered_tb) from None
File "/om2/user/hgazula/venvs/ss2.15/lib/python3.9/site-packages/tensorflow/python/eager/execute.py", line 53, in quick_execute
tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
TypeError: <tf.Tensor 'sample_resolution/Tile:0' shape=(None, 3) dtype=float32> is out of scope and cannot be used here. Use return values, explicit Python locals or TensorFlow collections to access it.
Please see https://www.tensorflow.org/guide/function#all_outputs_of_a_tffunction_must_be_return_values for more information.
<tf.Tensor 'sample_resolution/Tile:0' shape=(None, 3) dtype=float32> was defined here:
File "/net/vast-storage/scratch/vast/gablab/hgazula/SynthSeg/scripts/misc/synth_202408.py", line 247, in <module>
File "/net/vast-storage/scratch/vast/gablab/hgazula/SynthSeg/scripts/misc/synth_202408.py", line 137, in labels_to_image_model
File "/om2/user/hgazula/venvs/ss2.15/lib/python3.9/site-packages/keras/src/utils/traceback_utils.py", line 65, in error_handler
File "/om2/user/hgazula/venvs/ss2.15/lib/python3.9/site-packages/keras/src/engine/base_layer.py", line 1063, in __call__
File "/om2/user/hgazula/venvs/ss2.15/lib/python3.9/site-packages/keras/src/engine/base_layer.py", line 2593, in _functional_construction_call
File "/om2/user/hgazula/venvs/ss2.15/lib/python3.9/site-packages/keras/src/engine/base_layer.py", line 2439, in _keras_tensor_symbolic_call
File "/om2/user/hgazula/venvs/ss2.15/lib/python3.9/site-packages/keras/src/engine/base_layer.py", line 2498, in _infer_output_signature
File "/om2/user/hgazula/venvs/ss2.15/lib/python3.9/site-packages/keras/src/utils/traceback_utils.py", line 96, in error_handler
File "/om2/user/hgazula/SynthSeg/ext/lab2im/layers.py", line 715, in call
File "/om2/user/hgazula/venvs/ss2.15/lib/python3.9/site-packages/tensorflow/python/autograph/operators/control_flow.py", line 1217, in if_stmt
File "/om2/user/hgazula/venvs/ss2.15/lib/python3.9/site-packages/tensorflow/python/autograph/operators/control_flow.py", line 1270, in _py_if_stmt
File "/om2/user/hgazula/SynthSeg/ext/lab2im/layers.py", line 733, in call
File "/om2/user/hgazula/venvs/ss2.15/lib/python3.9/site-packages/tensorflow/python/ops/gen_array_ops.py", line 12045, in tile
File "/om2/user/hgazula/venvs/ss2.15/lib/python3.9/site-packages/tensorflow/python/framework/op_def_library.py", line 796, in _apply_op_helper
File "/om2/user/hgazula/venvs/ss2.15/lib/python3.9/site-packages/tensorflow/python/framework/func_graph.py", line 670, in _create_op_internal
File "/om2/user/hgazula/venvs/ss2.15/lib/python3.9/site-packages/tensorflow/python/framework/ops.py", line 2652, in _create_op_internal
File "/om2/user/hgazula/venvs/ss2.15/lib/python3.9/site-packages/tensorflow/python/framework/ops.py", line 1160, in from_node_def
The tensor <tf.Tensor 'sample_resolution/Tile:0' shape=(None, 3) dtype=float32> cannot be accessed from here, because it was defined in FuncGraph(name=sample_resolution_scratch_graph, id=47607497100160), which is out of scope.
I was wondering if you happen to have any ideas. I will also reach out to other Synth users to see if anyone's using the latest version of TF for their work.
Fixed. Thank you. See https://github.com/neuronets/nobrainer/issues/338#issuecomment-2282768104 for resolution. FYI, It is backward compatible as well.
Hi @BBillot
While the official recommended (according to the requirements.txt) TensorFlow is 2.2, I am trying to run SynthSeg in a more recent version of TensorFlow (constrained due to nobrainer) and it throws the following error. Admittedly, this is the case with almost every version from 2.3 to 2.15. 2.16 and above throws a different error which is for a different day.
Would you be kind enough to spare some time and help me fix this bug? I am happy to do any groundwork (environment setup, colab if you prefer) to make it easy for you. I'll be looking forward to your thoughts.
Note: The following error was thrown when running
SynthSeg/scripts/tutorials/1-generation_visualisation.py
in TF=2.15