Open thestarboy opened 4 years ago
yifanai
commented
4 years ago @thestarboy current repo is implemented with TensorFlow 1.14, because the original implementation uses instance norm layer only available in tf.contrib (which is removed in recent TF releases). Please try a running in a virtualenv with TF 1.14.
thestarboy
commented
4 years ago Same error even after install TF 1.14
yifanai
commented
4 years ago Seems like it's not reading the checkpoint files. Did you download the 3 checkpoint files and put them under checkpoints folder?
(0) Data loss: Unable to open table file checkpoints/UGATIT_100_epoch_generator_only: Failed precondition: checkpoints/UGATIT_100_epoch_generator_only; Is a directory: perhaps your file is in a different file format and you need to use a different restore operator? [[{{node save/RestoreV2}}]] [[save/RestoreV2/_149]] (1) Data loss: Unable to open table file checkpoints/UGATIT_100_epoch_generator_only: Failed precondition: checkpoints/UGATIT_100_epoch_generator_only; Is a directory: perhaps your file is in a different file format and you need to use a different restore operator? [[{{node save/RestoreV2}}]]
yifanai
commented
4 years ago By default, the 3 files should be just inside the checkpoints/ directory with no intermediate/subdirectory in between, like so:
checkpoints/
If you put them in another directory or subdirectory, you may specify at runtime: python selfie.py --checkpoint [CUSTOM DIR/CHECKPOINT PREFIX] (where prefix is UGATIT_100_epoch_generator_only in this case)
thestarboy
commented
4 years ago Name of folder containing pre-trained models was checkpoint now I have changed The name into checkpoints but still no luck. new error: cannot connect to X server
yifanai
commented
4 years ago Are you connecting to some remote server via SSH or Docker? The code uses OpenCV to draw/show results. Maybe this can help? https://stackoverflow.com/questions/58922358/python-cv2-imshow-not-working-cannot-connect-to-x-server
WARNING:tensorflow:From selfie.py:22: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.
WARNING:tensorflow:From /content/drive/My Drive/video2anime/UGATIT.py:12: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.
WARNING:tensorflow:From /tensorflow-1.15.2/python3.6/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.init (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version. Instructions for updating: If using Keras pass *_constraint arguments to layers. WARNING:tensorflow: The TensorFlow contrib module will not be included in TensorFlow 2.0. For more information, please see:
WARNING:tensorflow:From /content/drive/My Drive/video2anime/UGATIT.py:95: The name tf.get_variable is deprecated. Please use tf.compat.v1.get_variable instead.
WARNING:tensorflow:From selfie.py:26: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.
2020-04-26 04:37:46.442266: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2020-04-26 04:37:46.465864: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2020-04-26 04:37:46.466363: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1639] Found device 0 with properties: name: Tesla P4 major: 6 minor: 1 memoryClockRate(GHz): 1.1135 pciBusID: 0000:00:04.0 2020-04-26 04:37:46.466652: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2020-04-26 04:37:46.468439: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2020-04-26 04:37:46.470015: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2020-04-26 04:37:46.470384: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2020-04-26 04:37:46.472280: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2020-04-26 04:37:46.473437: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2020-04-26 04:37:46.477048: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2020-04-26 04:37:46.477158: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2020-04-26 04:37:46.477753: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2020-04-26 04:37:46.478196: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1767] Adding visible gpu devices: 0 2020-04-26 04:37:46.478506: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX512F 2020-04-26 04:37:46.483066: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2000155000 Hz 2020-04-26 04:37:46.483241: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x252cf40 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2020-04-26 04:37:46.483288: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2020-04-26 04:37:46.575660: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2020-04-26 04:37:46.576173: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x252d100 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2020-04-26 04:37:46.576201: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Tesla P4, Compute Capability 6.1 2020-04-26 04:37:46.576412: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2020-04-26 04:37:46.576778: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1639] Found device 0 with properties: name: Tesla P4 major: 6 minor: 1 memoryClockRate(GHz): 1.1135 pciBusID: 0000:00:04.0 2020-04-26 04:37:46.576850: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2020-04-26 04:37:46.576878: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2020-04-26 04:37:46.576901: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2020-04-26 04:37:46.576923: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2020-04-26 04:37:46.576943: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2020-04-26 04:37:46.576963: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2020-04-26 04:37:46.576983: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2020-04-26 04:37:46.577056: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2020-04-26 04:37:46.577502: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2020-04-26 04:37:46.577845: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1767] Adding visible gpu devices: 0 2020-04-26 04:37:46.577922: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2020-04-26 04:37:46.578962: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1180] Device interconnect StreamExecutor with strength 1 edge matrix: 2020-04-26 04:37:46.578988: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1186] 0 2020-04-26 04:37:46.578998: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1199] 0: N 2020-04-26 04:37:46.579107: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2020-04-26 04:37:46.579529: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2020-04-26 04:37:46.579880: W tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:39] Overriding allow_growth setting because the TF_FORCE_GPU_ALLOW_GROWTH environment variable is set. Original config value was 0. 2020-04-26 04:37:46.579921: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1325] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 7123 MB memory) -> physical GPU (device: 0, name: Tesla P4, pci bus id: 0000:00:04.0, compute capability: 6.1) WARNING:tensorflow:From selfie.py:27: The name tf.global_variables_initializer is deprecated. Please use tf.compat.v1.global_variables_initializer instead.
WARNING:tensorflow:From selfie.py:28: The name tf.train.Saver is deprecated. Please use tf.compat.v1.train.Saver instead.
2020-04-26 04:37:47.477417: W tensorflow/core/util/tensor_slice_reader.cc:95] Could not open checkpoints/UGATIT_100_epoch_generator_only: Failed precondition: checkpoints/UGATIT_100_epoch_generator_only; Is a directory: perhaps your file is in a different file format and you need to use a different restore operator? 2020-04-26 04:37:47.479156: W tensorflow/core/util/tensor_slice_reader.cc:95] Could not open checkpoints/UGATIT_100_epoch_generator_only: Failed precondition: checkpoints/UGATIT_100_epoch_generator_only; Is a directory: perhaps your file is in a different file format and you need to use a different restore operator? 2020-04-26 04:37:47.479199: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at save_restore_tensor.cc:175 : Data loss: Unable to open table file checkpoints/UGATIT_100_epoch_generator_only: Failed precondition: checkpoints/UGATIT_100_epoch_generator_only; Is a directory: perhaps your file is in a different file format and you need to use a different restore operator? Traceback (most recent call last): File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/client/session.py", line 1365, in _do_call return fn(*args) File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/client/session.py", line 1350, in _run_fn target_list, run_metadata) File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/client/session.py", line 1443, in _call_tf_sessionrun run_metadata) tensorflow.python.framework.errors_impl.DataLossError: 2 root error(s) found. (0) Data loss: Unable to open table file checkpoints/UGATIT_100_epoch_generator_only: Failed precondition: checkpoints/UGATIT_100_epoch_generator_only; Is a directory: perhaps your file is in a different file format and you need to use a different restore operator? [[{{node save/RestoreV2}}]] [[save/RestoreV2/_149]] (1) Data loss: Unable to open table file checkpoints/UGATIT_100_epoch_generator_only: Failed precondition: checkpoints/UGATIT_100_epoch_generator_only; Is a directory: perhaps your file is in a different file format and you need to use a different restore operator? [[{{node save/RestoreV2}}]] 0 successful operations. 0 derived errors ignored.
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "selfie.py", line 75, in
main(args)
File "selfie.py", line 29, in main
restorer.restore(sess, args.checkpoint)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/training/saver.py", line 1290, in restore
{self.saver_def.filename_tensor_name: save_path})
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/client/session.py", line 956, in run
run_metadata_ptr)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/client/session.py", line 1180, in _run
feed_dict_tensor, options, run_metadata)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/client/session.py", line 1359, in _do_run
run_metadata)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/client/session.py", line 1384, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.DataLossError: 2 root error(s) found.
(0) Data loss: Unable to open table file checkpoints/UGATIT_100_epoch_generator_only: Failed precondition: checkpoints/UGATIT_100_epoch_generator_only; Is a directory: perhaps your file is in a different file format and you need to use a different restore operator?
[[node save/RestoreV2 (defined at /tensorflow-1.15.2/python3.6/tensorflow_core/python/framework/ops.py:1748) ]]
[[save/RestoreV2/_149]]
(1) Data loss: Unable to open table file checkpoints/UGATIT_100_epoch_generator_only: Failed precondition: checkpoints/UGATIT_100_epoch_generator_only; Is a directory: perhaps your file is in a different file format and you need to use a different restore operator?
[[node save/RestoreV2 (defined at /tensorflow-1.15.2/python3.6/tensorflow_core/python/framework/ops.py:1748) ]]
0 successful operations.
0 derived errors ignored.
Original stack trace for 'save/RestoreV2': File "selfie.py", line 75, in
main(args)
File "selfie.py", line 28, in main
restorer = tf.train.Saver()
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/training/saver.py", line 828, in init
self.build()
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/training/saver.py", line 840, in build
self._build(self._filename, build_save=True, build_restore=True)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/training/saver.py", line 878, in _build
build_restore=build_restore)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/training/saver.py", line 508, in _build_internal
restore_sequentially, reshape)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/training/saver.py", line 328, in _AddRestoreOps
restore_sequentially)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/training/saver.py", line 575, in bulk_restore
return io_ops.restore_v2(filename_tensor, names, slices, dtypes)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/ops/gen_io_ops.py", line 1696, in restore_v2
name=name)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/framework/op_def_library.py", line 794, in _apply_op_helper
op_def=op_def)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/util/deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/framework/ops.py", line 3357, in create_op
attrs, op_def, compute_device)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/framework/ops.py", line 3426, in _create_op_internal
op_def=op_def)
File "/tensorflow-1.15.2/python3.6/tensorflow_core/python/framework/ops.py", line 1748, in init
self._traceback = tf_stack.extract_stack()