openvinotoolkit / openvino

OpenVINO™ is an open-source toolkit for optimizing and deploying AI inference
https://docs.openvino.ai
Apache License 2.0
7.06k stars 2.22k forks source link

[Bug] RuntimeError: [ GENERAL_ERROR ] #14448

Closed Veereshdammur closed 1 year ago

Veereshdammur commented 1 year ago

I have a custom model in Tensorflow (Efficientdet D4) that I exported to IR through the model optimizer successfully. Running the inference on the CPU works fine but in the MYRIAD device, it fails while loading the model due to operations.

Here is the error output.

openvino@66cdc03b3c8f:/opt/intel/openvino_2022.2.0.7713$ python3 /tmp/test_dir/src/pen_detector_ncs2.py 
Traceback (most recent call last):
  File "/tmp/test_dir/src/pen_detector_ncs2.py", line 15, in <module>
    compiled_model = ie.compile_model(model=model, device_name=DEVICE_NAME)
  File "/opt/intel/openvino/python/python3.8/openvino/runtime/ie_api.py", line 387, in compile_model
    super().compile_model(model, device_name, {} if config is None else config),
RuntimeError: [ GENERAL_ERROR ] 
/home/jenkins/agent/workspace/private-ci/ie/build-linux-ubuntu20/b/repos/openvino/src/plugins/intel_myriad/common/src/ngraph/transformations/dynamic_to_static_shape.cpp:198 DynamicToStaticShape transformation encountered dynamic node NonMaxSuppressionV5 of type NonMaxSuppression ver. 9, but only [Abs ver. 0, Ceiling ver. 0, Clamp ver. 0, Concat ver. 0, Convert ver. 0, Exp ver. 0, ExpGatherElements ver. 0, Floor ver. 0, Log ver. 0, MatMul ver. 0, ...] types are supported for dynamic nodes

I am using openvino_2022.2.0.7713 within the docker container built on the openvino/ubuntu20_dev image.
OS : Ubuntu 20.04 LTS

rkazants commented 1 year ago

@Veereshdammur, looks limitation of MYRIAD device that does not support dynamic shapes for NonMaxSuppression operation. Can you try to convert your model with input static shapes using --input or --input_shape MO options?

Best regards, Roman

rkazants commented 1 year ago

@Maxim-Doronin, @DariaMityagina, please comment on MYRIAD support of dynamism for NMS operation.

Thanks, Roman

Veereshdammur commented 1 year ago

@Veereshdammur, looks limitation of MYRIAD device that does not support dynamic shapes for NonMaxSuppression operation. Can you try to convert your model with input static shapes using --input or --input_shape MO options?

Best regards, Roman

This is the command output after converting the model by specifying the input and input_shape parameters.

(openvino_env) veeresh@veeresh-ubuntu:~/Downloads$ mo --input_model inference_graph_penid_det.pb --input_shape=[1,512,512,3] --input image_arrays
Model Optimizer arguments:
Common parameters:
    - Path to the Input Model:  /home/veeresh/Downloads/inference_graph_penid_det.pb
    - Path for generated IR:    /home/veeresh/Downloads/.
    - IR output name:   inference_graph_penid_det
    - Log level:    ERROR
    - Batch:    Not specified, inherited from the model
    - Input layers:     image_arrays
    - Output layers:    Not specified, inherited from the model
    - Input shapes:     [1,512,512,3]
    - Source layout:    Not specified
    - Target layout:    Not specified
    - Layout:   Not specified
    - Mean values:  Not specified
    - Scale values:     Not specified
    - Scale factor:     Not specified
    - Precision of IR:  FP32
    - Enable fusing:    True
    - User transformations:     Not specified
    - Reverse input channels:   False
    - Enable IR generation for fixed input shape:   False
    - Use the transformations config file:  None
Advanced parameters:
    - Force the usage of legacy Frontend of Model Optimizer for model conversion into IR:   False
    - Force the usage of new Frontend of Model Optimizer for model conversion into IR:  False
TensorFlow specific parameters:
    - Input model in text protobuf format:  False
    - Path to model dump for TensorBoard:   None
    - List of shared libraries with TensorFlow custom layers implementation:    None
    - Update the configuration file with input/output node names:   None
    - Use configuration file used to generate the model with Object Detection API:  None
    - Use the config file:  None
OpenVINO runtime found in:  /home/veeresh/openvino_env/lib/python3.8/site-packages/openvino
OpenVINO runtime version:   2022.2.0-7713-af16ea1d79a-releases/2022/2
Model Optimizer version:    2022.2.0-7713-af16ea1d79a-releases/2022/2
[ WARNING ]  Changing Const node 'data_mul_7070770709' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7071170713' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7084370845' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7071570717' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7081970821' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7072770729' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7079570797' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7073970741' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7077170773' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7075170753' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7076370765' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7076770769' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7078370785' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7078770789' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7079170793' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7080770809' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7081170813' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7081570817' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7083170833' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7083570837' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7083970841' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7098370985' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7085570857' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7095970961' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7086770869' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7093570937' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7087970881' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7091170913' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7089170893' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7090370905' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7090770909' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7092370925' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7092770929' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7093170933' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7094770949' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7095170953' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7095570957' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7097170973' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7097570977' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7097970981' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7112371125' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7099570997' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7109971101' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7100771009' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7107571077' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7101971021' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7105171053' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7103171033' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7104371045' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7104771049' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7106371065' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7106771069' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7107171073' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7108771089' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7109171093' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7109571097' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7111171113' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7111571117' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7111971121' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7126371265' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7113571137' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7123971241' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7114771149' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7121571217' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7115971161' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7119171193' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7117171173' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7118371185' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7118771189' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7120371205' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7120771209' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7121171213' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7122771229' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7123171233' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7123571237' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7125171253' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7125571257' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7125971261' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7140371405' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7127571277' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7137971381' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7128771289' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7135571357' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7129971301' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7133171333' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7131171313' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7132371325' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7132771329' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7134371345' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7134771349' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7135171353' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7136771369' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7137171373' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7137571377' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7139171393' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7139571397' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7139971401' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7154371545' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7141571417' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7151971521' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7142771429' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7149571497' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7143971441' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7147171473' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7145171453' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7146371465' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7146771469' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7148371485' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7148771489' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7149171493' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7150771509' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7151171513' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7151571517' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7153171533' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7153571537' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7153971541' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7191171913' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7155571557' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7183171833' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7156771569' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7175171753' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7157971581' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7167171673' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7159171593' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7166771669' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7174771749' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7182771829' data type from float64 to <class 'numpy.float32'> for Mul operation
[ WARNING ]  Changing Const node 'data_mul_7190771909' data type from float64 to <class 'numpy.float32'> for Mul operation
[ SUCCESS ] Generated IR version 11 model.
[ SUCCESS ] XML file: /home/veeresh/Downloads/inference_graph_penid_det.xml
[ SUCCESS ] BIN file: /home/veeresh/Downloads/inference_graph_penid_det.bin
[ SUCCESS ] Total execution time: 86.42 seconds. 
[ SUCCESS ] Memory consumed: 912 MB. 
[ INFO ] The model was converted to IR v11, the latest model format that corresponds to the source DL framework input/output format. While IR v11 is backwards compatible with OpenVINO Inference Engine API v1.0, please use API v2.0 (as of 2022.1) to take advantage of the latest improvements in IR v11.
Find more information about API v2.0 and IR v11 at https://docs.openvino.ai

and inferencing the model on the MYRIAD has caused the same error,

openvino@66cdc03b3c8f:/opt/intel/openvino_2022.2.0.7713$ python3 /tmp/test_dir/src/pen_detector_ncs2.py 
Traceback (most recent call last):
  File "/tmp/test_dir/src/pen_detector_ncs2.py", line 16, in <module>
    compiled_model = ie.compile_model(model=model, device_name=DEVICE_NAME)
  File "/opt/intel/openvino/python/python3.8/openvino/runtime/ie_api.py", line 387, in compile_model
    super().compile_model(model, device_name, {} if config is None else config),
RuntimeError: [ GENERAL_ERROR ] 
/home/jenkins/agent/workspace/private-ci/ie/build-linux-ubuntu20/b/repos/openvino/src/plugins/intel_myriad/common/src/ngraph/transformations/dynamic_to_static_shape.cpp:198 DynamicToStaticShape transformation encountered dynamic node NonMaxSuppressionV5 of type NonMaxSuppression ver. 9, but only [Abs ver. 0, Ceiling ver. 0, Clamp ver. 0, Concat ver. 0, Convert ver. 0, Exp ver. 0, ExpGatherElements ver. 0, Floor ver. 0, Log ver. 0, MatMul ver. 0, ...] types are supported for dynamic nodes
openvino@66cdc03b3c8f:/opt/intel/openvino_2022.2.0.7713$ 
hbalasu1 commented 1 year ago

Hi @Veereshdammur, apologies for the late response. After investigating, it looks like MO still performs dynamic shapes. I believe the model optimizer will setup dynamic dimensions if at least one of the starts, ends, steps or axes is not constant. Since MYRIAD does not support dynamic shapes, it will fail to load the model.

Can you have a look at your mode and check for the NonMaxSuppression ? Because that might be the reason model optimized is still setup as a Dynamic shape.

hbalasu1 commented 1 year ago

Hi @Veereshdammur can we close this issue ?

avitial commented 1 year ago

Closing this, I hope previous responses were sufficient to help you proceed. Feel free to reopen and ask additional questions related to this topic.

BalintVSE commented 1 year ago

Im still seeing this issue, the problem is with the NonMaximumSupression node creating a dynamic shaped output:

When loading the model with openvino model server the second dimension is listed as variable size:

[2023-04-03 07:33:19.701][1][modelmanager][info][modelinstance.cpp:479] Output name: output; mapping_name: output; shape: (1,[0~20],6); precision: FP32; layout: N... [2023-04-03 07:33:20.059][1][modelmanager][error][modelinstance.cpp:683] Cannot compile model into target device; error: [ GENERAL_ERROR ] /home/jenkins/agent/workspace/private-ci/ie/build-linux-ubuntu20/b/repos/openvino/src/plugins/intel_myriad/common/src/ngraph/transformations/dynamic_to_static_shape.cpp:198 DynamicToStaticShape transformation encountered dynamic node /end2end/NonMaxSuppression of type NonMaxSuppression ver. 9, but only [Abs ver. 0, Ceiling ver. 0, Clamp ver. 0, Concat ver. 0, Convert ver. 0, Exp ver. 0, ExpGatherElements ver. 0, Floor ver. 0, Log ver. 0, MatMul ver. 0, ...] types are supported for dynamic nodes; model: model_demo; version: 3; device: MYRIAD