Unable to Convert Human pose detection 3D model from ONNX to IR format

[aswin-datakalp]

HI Team ,

I am trying to replicate the human 3D pose detection model published here : https://github.com/openvinotoolkit/open_model_zoo/blob/master/models/public/human-pose-estimation-3d-0001/description/human-pose-estimation-3d-0001.md

For converting the native Pytorch model to ONNC format, the command that i used is :

sudo python3 pytorch_to_onnx.py --model-path=./public/human-pose-estimation-3d-0001/ --model-name=PoseEstimationWithMobileNet --model-param=is_convertible_by_mo=True --import-module=model --weights=./public/human-pose-estimation-3d-0001/human-pose-estimation-3d-0001.pth --input-shape=1,3,256,448 --input-names=data --output-names=features,heatmaps,pafs --output-file=./public/human-pose-estimation-3d-0001/human-pose-estimation-3d-0001.onnx

Converting from Pytorch to ONNX is successful

Next i am trying to convert ONNX model to OPENVINO IR format using the below command :

sudo python3 /opt/intel/openvino_2020.4.287/deployment_tools/model_optimizer/mo_onnx.py --data_type FP32 --input=data --mean_values=data[128.0,128.0,128.0] --scale_values=data[255.0,255.0,255.0] --output=features,heatmaps,pafs --input_model=/opt/intel/openvino_2020.3.194/deployment_tools/open_model_zoo/tools/downloader/public/human-pose-estimation-3d-0001/human-pose-estimation-3d-0001.onnx

This convertion throws the following error :

Exception occurred during running replacer "REPLACEMENT_ID" (<class 'extensions.front.user_data_repack.UserDataRepack'>): No node with name features.

Attached the screenshot of the same !

Plz do let me know what is gong wrong !!

[eaidova]

@aswin-datakalp do you able to look on your ONNX model throw network visualization tools e.g. netron? the error says that there is not layer with such name. It can be signal that output layers names are just different (e.g. it is possible if you use different pytorch/onnx versions then tests) or something went wrong during conversion

[eaidova]

@aswin-datakalp , BTW do you see the same errors using model_downloader and model_converter tools for these operations?

[aswin-datakalp]

HI @eaidova ,

Yes i loaded the model into Netron and tried visualising the layers and im able to see all 3 required layers namely , Features , heatmaps ,pafs

Attached the screenshot below.

Also tried converting the models using the downloader and converter blogs and even they throw the same error :

Exception occurred during running replacer "REPLACEMENT_ID" (<class 'extensions.front.user_data_repack.UserDataRepack'>): No node with name features.

Commands found from dry running ./converter.py --all --dry_run listed the follows and tried them too !

========= (DRY RUN) Converting human-pose-estimation-3d-0001 to ONNX Conversion to ONNX command: /usr/bin/python3 /opt/intel/openvino_2020.3.194/deployment_tools/open_model_zoo/tools/downloader/pytorch_to_onnx.py --model-path=/opt/intel/openvino_2020.3.194/deployment_tools/open_model_zoo/tools/downloader/public/human-pose-estimation-3d-0001 --model-name=PoseEstimationWithMobileNet --model-param=is_convertible_by_mo=True --import-module=model --weights=/opt/intel/openvino_2020.3.194/deployment_tools/open_model_zoo/tools/downloader/public/human-pose-estimation-3d-0001/human-pose-estimation-3d-0001.pth --input-shape=1,3,256,448 --input-names=data --output-names=features,heatmaps,pafs --output-file=/opt/intel/openvino_2020.3.194/deployment_tools/open_model_zoo/tools/downloader/public/human-pose-estimation-3d-0001/human-pose-estimation-3d-0001.onnx

========= (DRY RUN) Converting human-pose-estimation-3d-0001 to IR (FP16) Conversion command: /usr/bin/python3 -- /opt/intel/openvino_2020.4.287/deployment_tools/model_optimizer/mo.py --framework=onnx --data_type=FP16 --output_dir=/opt/intel/openvino_2020.3.194/deployment_tools/open_model_zoo/tools/downloader/public/human-pose-estimation-3d-0001/FP16 --model_name=human-pose-estimation-3d-0001 --input=data '--mean_values=data[128.0,128.0,128.0]' '--scale_values=data[255.0,255.0,255.0]' --output=features,heatmaps,pafs --input_model=/opt/intel/openvino_2020.3.194/deployment_tools/open_model_zoo/tools/downloader/public/human-pose-estimation-3d-0001/human-pose-estimation-3d-0001.onnx

========= (DRY RUN) Converting human-pose-estimation-3d-0001 to IR (FP32) Conversion command: /usr/bin/python3 --/opt/intel/openvino_2020.4.287/deployment_tools/model_optimizer/mo.py --framework=onnx --data_type=FP32 --output_dir=/opt/intel/openvino_2020.3.194/deployment_tools/open_model_zoo/tools/downloader/public/human-pose-estimation-3d-0001/FP32 --model_name=human-pose-estimation-3d-0001 --input=data '--mean_values=data[128.0,128.0,128.0]' '--scale_values=data[255.0,255.0,255.0]' --output=features,heatmaps,pafs --input_model=/opt/intel/openvino_2020.3.194/deployment_tools/open_model_zoo/tools/downloader/public/human-pose-estimation-3d-0001/human-pose-estimation-3d-0001.onnx

[eaidova]

@aswin-datakalp which onnx version installed on your machine? I also see that you run MO and model converter from different packages (openvino_2020.3.194 in downloader case and openvino_2020.4.287). Could you please try to use one package (openvino_2020.4.287) for both operations?

[eaidova]

possibly observed behaviour can be explained by this PR - https://github.com/openvinotoolkit/open_model_zoo/pull/1149

[aswin-datakalp]

Hi @eaidova ,

The onnx version is : '1.6.0' And yes tried with single version of openvino_2020.4.287 and still it didn't work !!

Pytorch version : '1.5.1' Also tried downgrading the pytorch to 1.4.x, still didnt work !!

Kindly suggest some work around to get the IR files for human pose estimate 3d 001 model or plz publish one and share them just like how its being done for 2D human pose estimation !!

[eaidova]

just for clarification, @aswin-datakalp do you try to regenerate onnx with 2020.4.287 package or just run dry-run?

[aswin-datakalp]

@eaidova , Regenerated ONNX from 2020.4.287 package and then tried converting the same !!

ONNX got generated but convertion to IR didnt happen !

[eaidova]

@aswin-datakalp unfortunatly, I can not reproduce your issue I install openvino 2020.4.287, install onnx and pytorch version which you said and run following commands:

downloader.py --name human-pose-estimation-3d-0001
converter.py --name human-pose-estimation-3d-0001

and IR was successfully converted

[aswin-datakalp]

@eaidova , Does converter.py do the pytorch_to_onnx convertsion also before converting to IR format ??

Also does this conversion follow the parsm listed in .yml file ?

https://github.com/openvinotoolkit/open_model_zoo/blob/master/models/public/human-pose-estimation-3d-0001/model.yml

[eaidova]

downloader log:

~/intel/openvino_2020.4.287/deployment_tools/open_model_zoo/tools/downloader$ python downloader.py --name human-pose-estimation-3d-0001
################|| Downloading models ||################

========== Downloading /home/automation/intel/openvino_2020.4.287/deployment_tools/open_model_zoo/tools/downloader/public/human-pose-estimation-3d-0001/human-pose-estimation-3d-0001.tar.gz
... 100%, 17990 KB, 16655 KB/s, 1 seconds passed

################|| Post-processing ||################

========== Unpacking /home/automation/intel/openvino_2020.4.287/deployment_tools/open_model_zoo/tools/downloader/public/human-pose-estimation-3d-0001/human-pose-estimation-3d-0001.tar.gz

converter log:

~/intel/openvino_2020.4.287/deployment_tools/open_model_zoo/tools/downloader$ python converter.py --name human-pose-estimation-3d-0001
========== Converting human-pose-estimation-3d-0001 to ONNX
Conversion to ONNX command: /home/automation/eaidova/anaconda3/bin/python /home/automation/intel/openvino_2020.4.287/deployment_tools/open_model_zoo/tools/downloader/pytorch_to_onnx.py --model-path=/home/automation/intel/openvino_2020.4.287/deployment_tools/open_model_zoo/tools/downloader/public/human-pose-estimation-3d-0001 --model-name=PoseEstimationWithMobileNet --model-param=is_convertible_by_mo=True --import-module=model --weights=/home/automation/intel/openvino_2020.4.287/deployment_tools/open_model_zoo/tools/downloader/public/human-pose-estimation-3d-0001/human-pose-estimation-3d-0001.pth --input-shape=1,3,256,448 --input-names=data --output-names=features,heatmaps,pafs --output-file=/home/automation/intel/openvino_2020.4.287/deployment_tools/open_model_zoo/tools/downloader/public/human-pose-estimation-3d-0001/human-pose-estimation-3d-0001.onnx

ONNX check passed successfully.

========== Converting human-pose-estimation-3d-0001 to IR (FP16)
Conversion command: /home/automation/eaidova/anaconda3/bin/python -- /home/automation/intel/openvino_2020.4.287/deployment_tools/model_optimizer/mo.py --framework=onnx --data_type=FP16 --output_dir=/home/automation/intel/openvino_2020.4.287/deployment_tools/open_model_zoo/tools/downloader/public/human-pose-estimation-3d-0001/FP16 --model_name=human-pose-estimation-3d-0001 --input=data '--mean_values=data[128.0,128.0,128.0]' '--scale_values=data[255.0,255.0,255.0]' --output=features,heatmaps,pafs --input_model=/home/automation/intel/openvino_2020.4.287/deployment_tools/open_model_zoo/tools/downloader/public/human-pose-estimation-3d-0001/human-pose-estimation-3d-0001.onnx

Model Optimizer arguments:
Common parameters:
    - Path to the Input Model:  /home/automation/intel/openvino_2020.4.287/deployment_tools/open_model_zoo/tools/downloader/public/human-pose-estimation-3d-0001/human-pose-estimation-3d-0001.onnx
    - Path for generated IR:    /home/automation/intel/openvino_2020.4.287/deployment_tools/open_model_zoo/tools/downloader/public/human-pose-estimation-3d-0001/FP16
    - IR output name:   human-pose-estimation-3d-0001
    - Log level:    ERROR
    - Batch:    Not specified, inherited from the model
    - Input layers:     data
    - Output layers:    features,heatmaps,pafs
    - Input shapes:     Not specified, inherited from the model
    - Mean values:  data[128.0,128.0,128.0]
    - Scale values:     data[255.0,255.0,255.0]
    - Scale factor:     Not specified
    - Precision of IR:  FP16
    - Enable fusing:    True
    - Enable grouped convolutions fusing:   True
    - Move mean values to preprocess section:   False
    - Reverse input channels:   False
ONNX specific parameters:
Model Optimizer version:    
[ WARNING ]  
Detected not satisfied dependencies:
    test-generator: installed: 0.1.2, required: == 0.1.1

Please install required versions of components or use install_prerequisites script
/home/automation/intel/openvino_2020.4.287/deployment_tools/model_optimizer/install_prerequisites/install_prerequisites_onnx.sh
Note that install_prerequisites scripts may install additional components.

[ SUCCESS ] Generated IR version 10 model.
[ SUCCESS ] XML file: /home/automation/intel/openvino_2020.4.287/deployment_tools/open_model_zoo/tools/downloader/public/human-pose-estimation-3d-0001/FP16/human-pose-estimation-3d-0001.xml
[ SUCCESS ] BIN file: /home/automation/intel/openvino_2020.4.287/deployment_tools/open_model_zoo/tools/downloader/public/human-pose-estimation-3d-0001/FP16/human-pose-estimation-3d-0001.bin
[ SUCCESS ] Total execution time: 10.56 seconds. 
[ SUCCESS ] Memory consumed: 204 MB. 

========== Converting human-pose-estimation-3d-0001 to IR (FP32)
Conversion command: /home/automation/eaidova/anaconda3/bin/python -- /home/automation/intel/openvino_2020.4.287/deployment_tools/model_optimizer/mo.py --framework=onnx --data_type=FP32 --output_dir=/home/automation/intel/openvino_2020.4.287/deployment_tools/open_model_zoo/tools/downloader/public/human-pose-estimation-3d-0001/FP32 --model_name=human-pose-estimation-3d-0001 --input=data '--mean_values=data[128.0,128.0,128.0]' '--scale_values=data[255.0,255.0,255.0]' --output=features,heatmaps,pafs --input_model=/home/automation/intel/openvino_2020.4.287/deployment_tools/open_model_zoo/tools/downloader/public/human-pose-estimation-3d-0001/human-pose-estimation-3d-0001.onnx

Model Optimizer arguments:
Common parameters:
    - Path to the Input Model:  /home/automation/intel/openvino_2020.4.287/deployment_tools/open_model_zoo/tools/downloader/public/human-pose-estimation-3d-0001/human-pose-estimation-3d-0001.onnx
    - Path for generated IR:    /home/automation/intel/openvino_2020.4.287/deployment_tools/open_model_zoo/tools/downloader/public/human-pose-estimation-3d-0001/FP32
    - IR output name:   human-pose-estimation-3d-0001
    - Log level:    ERROR
    - Batch:    Not specified, inherited from the model
    - Input layers:     data
    - Output layers:    features,heatmaps,pafs
    - Input shapes:     Not specified, inherited from the model
    - Mean values:  data[128.0,128.0,128.0]
    - Scale values:     data[255.0,255.0,255.0]
    - Scale factor:     Not specified
    - Precision of IR:  FP32
    - Enable fusing:    True
    - Enable grouped convolutions fusing:   True
    - Move mean values to preprocess section:   False
    - Reverse input channels:   False
ONNX specific parameters:
Model Optimizer version:    
[ WARNING ]  
Detected not satisfied dependencies:
    test-generator: installed: 0.1.2, required: == 0.1.1

Please install required versions of components or use install_prerequisites script
/home/automation/intel/openvino_2020.4.287/deployment_tools/model_optimizer/install_prerequisites/install_prerequisites_onnx.sh
Note that install_prerequisites scripts may install additional components.

[ SUCCESS ] Generated IR version 10 model.
[ SUCCESS ] XML file: /home/automation/intel/openvino_2020.4.287/deployment_tools/open_model_zoo/tools/downloader/public/human-pose-estimation-3d-0001/FP32/human-pose-estimation-3d-0001.xml
[ SUCCESS ] BIN file: /home/automation/intel/openvino_2020.4.287/deployment_tools/open_model_zoo/tools/downloader/public/human-pose-estimation-3d-0001/FP32/human-pose-estimation-3d-0001.bin
[ SUCCESS ] Total execution time: 10.55 seconds. 
[ SUCCESS ] Memory consumed: 202 MB.

yes, these tools use parameters listed in model.yml

[aswin-datakalp]

@eaidova,

Looks like openvino_2020.3.194, has some issue when converting to IR format.

Now i am able to get the IR format models when i completely remove openvino_2020.3.194 by doing sudo apt autoremove and installed openvino_2020.4.x .

Thanks for your timely help !!