Open BWWS opened 2 years ago
install 'tf-models-official==2.7.1',
you can also change the script: Tensorflow/models/research/object_detection/packages/tf2/setup.py
'tf-models-official==2.7.1',
Hi SauBuen, Thanks but did not work fully. Passes the verification script but now fails at the training script:
python Tensorflow/models/research/object_detection/model_main_tf2.py --model_dir=Tensorflow/workspace/models/Seep_ssd_mobnet --pipeline_config_path=Tensorflow/workspace/models/Seep_ssd_mobnet/pipeline.config --num_train_steps=2000
2022-02-08 10:31:17.000189: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-02-08 10:31:17.009763: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-02-08 10:31:17.010590: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-02-08 10:31:17.011815: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-02-08 10:31:17.012111: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-02-08 10:31:17.012903: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-02-08 10:31:17.013704: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-02-08 10:31:17.570257: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-02-08 10:31:17.571145: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-02-08 10:31:17.571903: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-02-08 10:31:17.572642: W tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:39] Overriding allow_growth setting because the TF_FORCE_GPU_ALLOW_GROWTH environment variable is set. Original config value was 0.
2022-02-08 10:31:17.572721: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 10663 MB memory: -> device: 0, name: Tesla K80, pci bus id: 0000:00:04.0, compute capability: 3.7
INFO:tensorflow:Using MirroredStrategy with devices ('/job:localhost/replica:0/task:0/device:GPU:0',)
I0208 10:31:17.578470 139906285909888 mirrored_strategy.py:369] Using MirroredStrategy with devices ('/job:localhost/replica:0/task:0/device:GPU:0',)
INFO:tensorflow:Maybe overwriting train_steps: 2000
I0208 10:31:17.583169 139906285909888 config_util.py:552] Maybe overwriting train_steps: 2000
INFO:tensorflow:Maybe overwriting use_bfloat16: False
I0208 10:31:17.583358 139906285909888 config_util.py:552] Maybe overwriting use_bfloat16: False
WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/object_detection/model_lib_v2.py:564: StrategyBase.experimental_distribute_datasets_from_function (from tensorflow.python.distribute.distribute_lib) is deprecated and will be removed in a future version.
Instructions for updating:
rename to distribute_datasets_from_function
W0208 10:31:17.613617 139906285909888 deprecation.py:345] From /usr/local/lib/python3.7/dist-packages/object_detection/model_lib_v2.py:564: StrategyBase.experimental_distribute_datasets_from_function (from tensorflow.python.distribute.distribute_lib) is deprecated and will be removed in a future version.
Instructions for updating:
rename to distribute_datasets_from_function
INFO:tensorflow:Reading unweighted datasets: ['Tensorflow/workspace/annotations/train.record']
I0208 10:31:17.618627 139906285909888 dataset_builder.py:163] Reading unweighted datasets: ['Tensorflow/workspace/annotations/train.record']
INFO:tensorflow:Reading record datasets for input file: ['Tensorflow/workspace/annotations/train.record']
I0208 10:31:17.618862 139906285909888 dataset_builder.py:80] Reading record datasets for input file: ['Tensorflow/workspace/annotations/train.record']
INFO:tensorflow:Number of filenames to read: 1
I0208 10:31:17.619039 139906285909888 dataset_builder.py:81] Number of filenames to read: 1
WARNING:tensorflow:num_readers has been reduced to 1 to match input file shards.
W0208 10:31:17.619182 139906285909888 dataset_builder.py:88] num_readers has been reduced to 1 to match input file shards.
WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/object_detection/builders/dataset_builder.py:105: parallel_interleave (from tensorflow.python.data.experimental.ops.interleave_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.data.Dataset.interleave(map_func, cycle_length, block_length, num_parallel_calls=tf.data.AUTOTUNE)
instead. If sloppy execution is desired, use tf.data.Options.experimental_deterministic
.
W0208 10:31:17.622419 139906285909888 deprecation.py:345] From /usr/local/lib/python3.7/dist-packages/object_detection/builders/dataset_builder.py:105: parallel_interleave (from tensorflow.python.data.experimental.ops.interleave_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.data.Dataset.interleave(map_func, cycle_length, block_length, num_parallel_calls=tf.data.AUTOTUNE)
instead. If sloppy execution is desired, use tf.data.Options.experimental_deterministic
.
WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/object_detection/builders/dataset_builder.py:237: DatasetV1.map_with_legacy_function (from tensorflow.python.data.ops.dataset_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.data.Dataset.map() W0208 10:31:17.646344 139906285909888 deprecation.py:345] From /usr/local/lib/python3.7/dist-packages/object_detection/builders/dataset_builder.py:237: DatasetV1.map_with_legacy_function (from tensorflow.python.data.ops.dataset_ops) is deprecated and will be removed in a future version. Instructions for updating: Use
tf.data.Dataset.map()
WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow/python/util/dispatch.py:206: sparse_to_dense (from tensorflow.python.ops.sparse_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Create a tf.sparse.SparseTensor
and use tf.sparse.to_dense
instead.
W0208 10:31:26.506816 139906285909888 deprecation.py:345] From /usr/local/lib/python3.7/dist-packages/tensorflow/python/util/dispatch.py:206: sparse_to_dense (from tensorflow.python.ops.sparse_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Create a tf.sparse.SparseTensor
and use tf.sparse.to_dense
instead.
WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow/python/util/dispatch.py:206: sample_distorted_bounding_box (from tensorflow.python.ops.image_ops_impl) is deprecated and will be removed in a future version.
Instructions for updating:
seed2
arg is deprecated.Use sample_distorted_bounding_box_v2 instead.
W0208 10:31:30.294101 139906285909888 deprecation.py:345] From /usr/local/lib/python3.7/dist-packages/tensorflow/python/util/dispatch.py:206: sample_distorted_bounding_box (from tensorflow.python.ops.image_ops_impl) is deprecated and will be removed in a future version.
Instructions for updating:
seed2
arg is deprecated.Use sample_distorted_bounding_box_v2 instead.
WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow/python/autograph/impl/api.py:464: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast
instead.
W0208 10:31:32.181428 139906285909888 deprecation.py:345] From /usr/local/lib/python3.7/dist-packages/tensorflow/python/autograph/impl/api.py:464: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast
instead.
2022-02-08 10:31:34.971475: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:185] None of the MLIR Optimization Passes are enabled (registered 2)
/usr/local/lib/python3.7/dist-packages/keras/backend.py:401: UserWarning: tf.keras.backend.set_learning_phase
is deprecated and will be removed after 2020-10-11. To update it, simply pass a True/False value to the training
argument of the __call__
method of your layer or model.
warnings.warn('tf.keras.backend.set_learning_phase
is deprecated and '
2022-02-08 10:31:58.458210: E tensorflow/stream_executor/cuda/cuda_dnn.cc:362] Loaded runtime CuDNN library: 8.0.5 but source was compiled with: 8.1.0. CuDNN library needs to have matching major version and equal or higher minor version. If using a binary install, upgrade your CuDNN library. If building from sources, make sure the library loaded at runtime is compatible with the version specified during compile configuration.
2022-02-08 10:31:58.461067: E tensorflow/stream_executor/cuda/cuda_dnn.cc:362] Loaded runtime CuDNN library: 8.0.5 but source was compiled with: 8.1.0. CuDNN library needs to have matching major version and equal or higher minor version. If using a binary install, upgrade your CuDNN library. If building from sources, make sure the library loaded at runtime is compatible with the version specified during compile configuration.
Traceback (most recent call last):
File "Tensorflow/models/research/object_detection/model_main_tf2.py", line 115, in
Errors may have originated from an input operation. Input Source operations connected to node ssd_mobile_net_v2fpn_keras_feature_extractor/model/Conv1/Conv2D: args_1 (defined at /usr/local/lib/python3.7/dist-packages/object_detection/model_lib_v2.py:178)
Function call stack: _dummy_computation_fn
Now I am having the same issue, before it was working
I asked this same question in the TensorFlow models guithub issue#10505 I think there it is more probable to get help there
I am trying to run the code direct from uploading into COLAB from GitHub I am unable to get past the VERIFICATION_SCRIPT
It reports the following error:
Traceback (most recent call last): File "Tensorflow/models/research/object_detection/builders/model_builder_tf2_test.py", line 25, in
from object_detection.builders import model_builder
File "/usr/local/lib/python3.7/dist-packages/object_detection/builders/model_builder.py", line 37, in
from object_detection.meta_architectures import deepmac_meta_arch
File "/usr/local/lib/python3.7/dist-packages/object_detection/meta_architectures/deepmac_meta_arch.py", line 19, in
from object_detection.models.keras_models import resnet_v1
File "/usr/local/lib/python3.7/dist-packages/object_detection/models/keras_models/resnet_v1.py", line 22, in
from tensorflow.python.keras.applications import resnet
ModuleNotFoundError: No module named 'tensorflow.python.keras.applications'
Using the latest Colab versions:
tensorflow 2.8.0 tensorflow-io 0.23.1 keras 2.8.0 Keras-Preprocessing 1.1.2
I have tried using older versions ranging from 2.3.0 but these just lead to different errors
Please advise if you have a solution to this.