Closed goldwater668 closed 4 years ago
I also tested inference on RTX 2080 Ti while getting no bounding boxes, but I got bounding boxes with RTX 2080.
After I checked out 3d88847cc18c69d194490f039279502ddcb536f2, it works again. There are some breaking changes have been introduced in the project by later commits.
Good to know it works now. closing this issue for now.
2020-03-27 20:29:13.328542: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libnvinfer.so.6 2020-03-27 20:29:13.329598: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libnvinfer_plugin.so.6 WARNING:tensorflow:From /home/hjq/anaconda3/envs/efficientDet/lib/python3.6/site-packages/tensorflow_core/python/compat/v2_compat.py:88: disable_resource_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in a future version. Instructions for updating: non-resource variables are not supported in the long term 2020-03-27 20:29:13.955696: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2020-03-27 20:29:13.980297: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2020-03-27 20:29:13.980864: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1555] Found device 0 with properties: pciBusID: 0000:01:00.0 name: GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2020-03-27 20:29:13.980941: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2020-03-27 20:29:13.981017: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2020-03-27 20:29:13.981766: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2020-03-27 20:29:13.981974: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2020-03-27 20:29:13.982926: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2020-03-27 20:29:13.983663: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2020-03-27 20:29:13.983743: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2020-03-27 20:29:13.983807: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2020-03-27 20:29:13.984336: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2020-03-27 20:29:13.984800: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1697] Adding visible gpu devices: 0 2020-03-27 20:29:13.985040: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2020-03-27 20:29:14.007984: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3600000000 Hz 2020-03-27 20:29:14.008362: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x39b6400 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2020-03-27 20:29:14.008545: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2020-03-27 20:29:14.088095: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2020-03-27 20:29:14.088587: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x392d960 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2020-03-27 20:29:14.088724: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): GeForce RTX 2080 Ti, Compute Capability 7.5 2020-03-27 20:29:14.088890: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2020-03-27 20:29:14.089295: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1555] Found device 0 with properties: pciBusID: 0000:01:00.0 name: GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2020-03-27 20:29:14.089370: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2020-03-27 20:29:14.089437: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2020-03-27 20:29:14.089510: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2020-03-27 20:29:14.089577: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2020-03-27 20:29:14.089645: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2020-03-27 20:29:14.089669: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2020-03-27 20:29:14.089691: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2020-03-27 20:29:14.089735: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2020-03-27 20:29:14.090080: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2020-03-27 20:29:14.090402: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1697] Adding visible gpu devices: 0 2020-03-27 20:29:14.090434: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2020-03-27 20:29:14.250949: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1096] Device interconnect StreamExecutor with strength 1 edge matrix: 2020-03-27 20:29:14.251062: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] 0 2020-03-27 20:29:14.251242: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] 0: N 2020-03-27 20:29:14.251549: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2020-03-27 20:29:14.252163: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2020-03-27 20:29:14.252682: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1241] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10121 MB memory) -> physical GPU (device: 0, name: GeForce RTX 2080 Ti, pci bus id: 0000:01:00.0, compute capability: 7.5) WARNING:tensorflow:From /data/internet/AI/automl/efficientdet/dataloader.py:100: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use
tf.cast
instead. W0327 20:29:14.281555 140386854647616 deprecation.py:323] From /data/internet/AI/automl/efficientdet/dataloader.py:100: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Usetf.cast
instead. WARNING:tensorflow:From /data/internet/AI/automl/efficientdet/dataloader.py:102: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Usetf.cast
instead. W0327 20:29:14.283189 140386854647616 deprecation.py:323] From /data/internet/AI/automl/efficientdet/dataloader.py:102: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Usetf.cast
instead. WARNING:tensorflow:From /home/hjq/anaconda3/envs/efficientDet/lib/python3.6/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1635: calling BaseResourceVariable.init (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version. Instructions for updating: If using Keras pass _constraint arguments to layers. W0327 20:29:15.400412 140386854647616 deprecation.py:506] From /home/hjq/anaconda3/envs/efficientDet/lib/python3.6/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1635: calling BaseResourceVariable.init (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version. Instructions for updating: If using Keras pass _constraint arguments to layers. WARNING:tensorflow:From /home/hjq/anaconda3/envs/efficientDet/lib/python3.6/site-packages/tensorflow_core/python/profiler/internal/flops_registry.py:142: tensor_shape_from_node_def_name (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version. Instructions for updating: Usetf.compat.v1.graph_util.tensor_shape_from_node_def_name
W0327 20:29:17.909279 140386854647616 deprecation.py:323] From /home/hjq/anaconda3/envs/efficientDet/lib/python3.6/site-packages/tensorflow_core/python/profiler/internal/flops_registry.py:142: tensor_shape_from_node_def_name (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version. Instructions for updating: Usetf.compat.v1.graph_util.tensor_shape_from_node_def_name
WARNING:tensorflow:From /data/internet/AI/automl/efficientdet/efficientdet_arch.py:94: conv2d (from tensorflow.python.layers.convolutional) is deprecated and will be removed in a future version. Instructions for updating: Usetf.keras.layers.Conv2D
instead. W0327 20:29:18.113473 140386854647616 deprecation.py:323] From /data/internet/AI/automl/efficientdet/efficientdet_arch.py:94: conv2d (from tensorflow.python.layers.convolutional) is deprecated and will be removed in a future version. Instructions for updating: Usetf.keras.layers.Conv2D
instead. WARNING:tensorflow:From /home/hjq/anaconda3/envs/efficientDet/lib/python3.6/site-packages/tensorflow_core/python/layers/convolutional.py:424: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version. Instructions for updating: Please uselayer.__call__
method instead. W0327 20:29:18.114022 140386854647616 deprecation.py:323] From /home/hjq/anaconda3/envs/efficientDet/lib/python3.6/site-packages/tensorflow_core/python/layers/convolutional.py:424: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version. Instructions for updating: Please uselayer.__call__
method instead. WARNING:tensorflow:From /data/internet/AI/automl/efficientdet/efficientdet_arch.py:121: max_pooling2d (from tensorflow.python.layers.pooling) is deprecated and will be removed in a future version. Instructions for updating: Use keras.layers.MaxPooling2D instead. W0327 20:29:18.137391 140386854647616 deprecation.py:323] From /data/internet/AI/automl/efficientdet/efficientdet_arch.py:121: max_pooling2d (from tensorflow.python.layers.pooling) is deprecated and will be removed in a future version. Instructions for updating: Use keras.layers.MaxPooling2D instead. WARNING:tensorflow:From /data/internet/AI/automl/efficientdet/efficientdet_arch.py:516: separable_conv2d (from tensorflow.python.layers.convolutional) is deprecated and will be removed in a future version. Instructions for updating: Usetf.keras.layers.SeparableConv2D
instead. W0327 20:29:18.148901 140386854647616 deprecation.py:323] From /data/internet/AI/automl/efficientdet/efficientdet_arch.py:516: separable_conv2d (from tensorflow.python.layers.convolutional) is deprecated and will be removed in a future version. Instructions for updating: Usetf.keras.layers.SeparableConv2D
instead. WARNING:tensorflow:From /home/hjq/anaconda3/envs/efficientDet/lib/python3.6/site-packages/tensorflow_core/python/training/moving_averages.py:433: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version. Instructions for updating: Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts. W0327 20:29:21.665508 140386854647616 deprecation.py:323] From /home/hjq/anaconda3/envs/efficientDet/lib/python3.6/site-packages/tensorflow_core/python/training/moving_averages.py:433: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version. Instructions for updating: Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts. WARNING:tensorflow:From /data/internet/AI/automl/efficientdet/det_model_fn.py:316: div (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Deprecated in favor of operator or tf.math.divide. W0327 20:29:32.980791 140386854647616 deprecation.py:323] From /data/internet/AI/automl/efficientdet/det_model_fn.py:316: div (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Deprecated in favor of operator or tf.math.divide. WARNING:tensorflow:From /data/internet/AI/automl/efficientdet/anchors.py:398: py_func (from tensorflow.python.ops.script_ops) is deprecated and will be removed in a future version. Instructions for updating: tf.py_func is deprecated in TF V2. Instead, there are two options available in V2.tf.py_function
s can use accelerators such as GPUs as well as being differentiable using a gradient tape.W0327 20:29:32.993462 140386854647616 deprecation.py:323] From /data/internet/AI/automl/efficientdet/anchors.py:398: py_func (from tensorflow.python.ops.script_ops) is deprecated and will be removed in a future version. Instructions for updating: tf.py_func is deprecated in TF V2. Instead, there are two options available in V2.
tf.py_function
s can use accelerators such as GPUs as well as being differentiable using a gradient tape.2020-03-27 20:29:34.324642: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2020-03-27 20:29:35.100379: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 ckpt_path: /data/internet/AI/automl/efficientdet/ckpt/efficientdet-d0/ Parsing Inputs... Parsing Inputs... Parsing Inputs...