Unable to Export ST-GCN Model to ONNX using export_onnx_stdet.py script

gorkanauta commented 1 year ago

Branch

main branch (1.x version, such as v1.0.0, or dev-1.x branch)

Prerequisite

[X] I have searched Issues and Discussions but cannot get the expected help.
[X] I have read the documentation but cannot get the expected help.
[X] The bug has not been fixed in the latest version.

Environment

OS: Linux 8331746310b5 5.4.0-1104-gcp Python: PyTorch: 2.0.1+cu117 CUDA: 11.7 GCC: gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0 PyTorch compiling details: sh: 1: python: not found PyTorch built with:

GCC 9.3
C++ Version: 201703
Intel(R) oneAPI Math Kernel Library Version 2022.2-Product Build 20220804 for Intel(R) 64 architecture applications
Intel(R) MKL-DNN v2.7.3 (Git Hash 6dbeffbae1f23cbbeae17adb7b5b13f1f37c080e)
OpenMP 201511 (a.k.a. OpenMP 4.5)
LAPACK is enabled (usually provided by MKL)
NNPACK is enabled
CPU capability usage: AVX2
CUDA Runtime 11.7
NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86
CuDNN 8.5
Magma 2.6.1
Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.7, CUDNN_VERSION=8.5.0

MMCV: 2.0.0 MMEngine: 0.7.3 MMAction2: 1.0.0

Describe the bug

Hello,

I have been trying to export an ST-GCN model trained on skeleton data to ONNX using the export_onnx_stdet.py script found in the MMAction2 repository, but I have encountered an issue.

My model takes in 3D skeleton data where the input shape consists of 51 frames, 17 joints and each joint has x, y, and score components. I am running the script on a GPU ("cuda:0"). The paths to my config file, checkpoint file and the script are as follows:

Config path: /workspaces/mmaction2-sng/configs/skeleton/stgcn/stgcn_8xb16-joint-u100-80e_ntu60-xsub-keypoint-2d.py Checkpoint path: /workspaces/mmaction2-sng/work_dirs/stgcn_8xb16-joint-u100-80e_ntu60-xsub-keypoint-2d/epoch_16.pth Script path: /workspaces/mmaction2-sng/tools/deployment/export_onnx_stdet.py Output file: action_detector.onnx When running the script, I encountered the following error: AttributeError: 'RecognizerGCN' object has no attribute 'roi_head'

From my understanding, this is because the script was designed for models that use region-based detectors (thus the 'roi_head'), but the ST-GCN model is a graph-based model that doesn't need to detect regions of interest.

Could you please provide some guidance on how to export an ST-GCN model to ONNX using your library? Any help would be greatly appreciated.

Thank you!

Reproduces the problem - command or script

python /workspaces/mmaction2-sng/tools/deployment/export_onnx_stdet.py /workspaces/mmaction2-sng/configs/skeleton/stgcn/stgcn_8xb16-joint-u100-80e_ntu60-xsub-keypoint-2d.py /workspaces/mmaction2-sng/work_dirs/stgcn_8xb16-joint-u100-80e_ntu60-xsub-keypoint-2d/epoch_16.pth --num_frames 51 --shape 17 3 --device cuda:0 --output_file action_detector.onnx

Reproduces the problem - error message

AttributeError: 'RecognizerGCN' object has no attribute 'roi_head'

hukkai commented 1 year ago

@gorkanauta export_onnx_stdet.py is used to export spatial-temporal detection models. ST-GCN is not supported now. You may do it by yourself, but we will also release soon. Please stay tuned.

gorkanauta commented 1 year ago

Any advice on how to do it by myself? Thank you!

gorkanauta commented 1 year ago

I did manage to export modifying the script, the exported model in onnx outputs values not normalized between 0 and 1 for each action. This is a sample output for the 5 class classifier I exported:

[array([[ 81.58998, 40.22279, -465.58243, 148.7499 , 194.898 ]], dtype=float32)]

  Is this normal? Should I apply a sigmoid afterwards? Or is this an issue?

open-mmlab / mmaction2