search

PINTO0309 / openvino2tensorflow

This script converts the ONNX/OpenVINO IR model to Tensorflow's saved_model, tflite, h5, tfjs, tftrt(TensorRT), CoreML, EdgeTPU, ONNX and pb. PyTorch (NCHW) -> ONNX (NCHW) -> OpenVINO (NCHW) -> openvino2tensorflow -> Tensorflow/Keras (NHWC/NCHW) -> TFLite (NHWC/NCHW). And the conversion from .pb to saved_model and from saved_model to .pb and from .pb to .tflite and saved_model to .tflite and saved_model to onnx. Support for building environments with Docker. It is possible to directly access the host PC GUI and the camera to verify the operation. NVIDIA GPU (dGPU) support. Intel iHD GPU (iGPU) support.
MIT License
332 stars 40 forks source link

Inconsistent model output after converting to tensorflow #109

Closed zye1996 closed 2 years ago

zye1996 commented 2 years ago

Issue Type

Bug

OS

Ubuntu

OS architecture

x86_64

Programming Language

Python

Framework

TensorFlow

Download URL for ONNX / OpenVINO IR

https://drive.google.com/file/d/1RufrET4Tz20FQNlFaCXCVwwiChOCS7ef/view?usp=sharing

Convert Script

python3 openvino2tensorflow/openvino2tensorflow2.py --model_path ../../test_data/iresnet50_relu/model.xml --model_output_path ../../test_data/iresnet50_relu/ --output_saved_model --output_no_quant_float32_tflite

Description

The onnx to openvino works ok and I verified output. But tensorflow output not expected

Relevant Log Output

/home/yzy/anaconda3/envs/nvr/bin/python /home/yzy/PycharmProjects/nvr/models/a311d/facerec/check_result.py
2022-06-02 21:09:37.677247: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudart.so.11.0
2022-06-02 21:09:39.517362: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcuda.so.1
2022-06-02 21:09:39.548373: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-06-02 21:09:39.548481: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1733] Found device 0 with properties: 
pciBusID: 0000:07:00.0 name: NVIDIA GeForce RTX 3090 computeCapability: 8.6
coreClock: 1.695GHz coreCount: 82 deviceMemorySize: 23.70GiB deviceMemoryBandwidth: 871.81GiB/s
2022-06-02 21:09:39.548503: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudart.so.11.0
2022-06-02 21:09:39.549826: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcublas.so.11
2022-06-02 21:09:39.549861: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcublasLt.so.11
2022-06-02 21:09:39.550423: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcufft.so.10
2022-06-02 21:09:39.550563: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcurand.so.10
2022-06-02 21:09:39.551943: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcusolver.so.11
2022-06-02 21:09:39.552236: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcusparse.so.11
2022-06-02 21:09:39.552327: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudnn.so.8
2022-06-02 21:09:39.552411: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-06-02 21:09:39.552532: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-06-02 21:09:39.552607: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1871] Adding visible gpu devices: 0
2022-06-02 21:09:39.552782: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-06-02 21:09:39.553525: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-06-02 21:09:39.553607: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1733] Found device 0 with properties: 
pciBusID: 0000:07:00.0 name: NVIDIA GeForce RTX 3090 computeCapability: 8.6
coreClock: 1.695GHz coreCount: 82 deviceMemorySize: 23.70GiB deviceMemoryBandwidth: 871.81GiB/s
2022-06-02 21:09:39.553647: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-06-02 21:09:39.553744: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-06-02 21:09:39.553812: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1871] Adding visible gpu devices: 0
2022-06-02 21:09:39.553833: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudart.so.11.0
2022-06-02 21:09:39.957899: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1258] Device interconnect StreamExecutor with strength 1 edge matrix:
2022-06-02 21:09:39.957925: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1264]      0 
2022-06-02 21:09:39.957930: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1277] 0:   N 
2022-06-02 21:09:39.958108: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-06-02 21:09:39.958246: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-06-02 21:09:39.958352: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-06-02 21:09:39.958443: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1418] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 21638 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 3090, pci bus id: 0000:07:00.0, compute capability: 8.6)
2022-06-02 21:09:42.811016: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:176] None of the MLIR Optimization Passes are enabled (registered 2)
2022-06-02 21:09:42.844458: I tensorflow/core/platform/profile_utils/cpu_utils.cc:114] CPU Frequency: 3700385000 Hz
2022-06-02 21:09:43.235369: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudnn.so.8
2022-06-02 21:09:43.700112: I tensorflow/stream_executor/cuda/cuda_dnn.cc:359] Loaded cuDNN version 8100
2022-06-02 21:09:44.225747: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcublas.so.11
2022-06-02 21:09:44.586999: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcublasLt.so.11
2022-06-02 21:09:44.716148: I tensorflow/stream_executor/cuda/cuda_blas.cc:1838] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
tf tf.Tensor(
[[ 5.99430323e-01  1.02042861e-01 -1.38508588e-01 -8.28384161e-02
  -1.15059829e+00 -3.07134449e-01  5.69472253e-01  3.97508711e-01
  -2.24888653e-01  9.62444186e-01  3.98875564e-01 -6.01023018e-01
  -3.34999204e-01  5.52027076e-02  2.58950949e-01  8.24933052e-01
  -3.71319532e-01  1.30895406e-01 -6.14488050e-02  3.68052214e-01
  -6.39150918e-01 -5.06587088e-01 -6.92968905e-01 -5.23465574e-02
  -2.33985022e-01  4.19633418e-01 -1.75420597e-01  1.35106206e-01
   1.37400493e-01  3.71858537e-01 -7.57282615e-01 -1.65140957e-01
  -4.83937800e-01  1.56087726e-01  1.08637370e-01  5.09591818e-01
  -2.41889298e-01  6.74896762e-02 -5.54498553e-01 -2.77842313e-01
  -1.58023775e-01 -1.44336075e-01  9.16118100e-02  1.91489160e-01
  -7.41642296e-01  3.41407321e-02 -6.00127995e-01 -2.28546545e-01
  -4.45287436e-01  2.73571163e-01  5.42080589e-02 -2.78937221e-01
   2.71262854e-01 -7.56002605e-01  3.21306646e-01  5.30472815e-01
  -2.16525123e-02  4.46718097e-01 -2.13284045e-01  3.48609984e-01
   1.68747157e-01  1.40319407e-01 -4.20820862e-01  3.39960158e-02
  -1.64804757e-01 -4.67031062e-01 -4.99531060e-01  1.74518123e-01
   2.19151452e-02 -6.14941642e-02 -4.61608171e-03 -9.56363976e-01
   3.40975583e-01  6.65674061e-02 -1.76031083e-01 -1.75278112e-01
   1.61821783e-01  2.77785361e-01 -3.21783900e-01 -2.15370029e-01
   2.91443527e-01 -1.39670283e-01  1.86092034e-01 -3.67239684e-01
   4.58371580e-01  2.26971194e-01  2.88200229e-01 -4.61214222e-02
   3.52189481e-01 -3.29671144e-01  2.04508856e-01  2.30899334e-01
   2.81684071e-01  5.43728888e-01  3.33083183e-01 -2.42001563e-02
  -1.28006354e-01  3.75516057e-01  2.54608750e-01  2.48512477e-01
  -1.57832801e-01  1.24746352e-01  2.64458418e-01  1.92484558e-01
   1.71757907e-01  2.72231907e-01  3.77800167e-01 -7.74545819e-02
  -8.55362773e-01 -3.43645990e-01  9.42596793e-02  1.18445896e-01
   4.12560329e-02  5.19879684e-02 -4.32934523e-01  5.56093633e-01
   5.69096208e-02  2.88947642e-01  3.64391029e-01  1.77076250e-01
   4.31404382e-01 -1.54377699e-01  6.07027411e-01  3.29601675e-01
  -6.40743375e-02 -2.94263124e-01 -2.36535013e-01 -5.47802091e-01
   1.37124658e-01  9.51397866e-02  1.08095288e-01 -6.82768375e-02
  -1.04074344e-01  5.60706258e-01  2.05955997e-01 -3.62044215e-01
  -5.35062402e-02  9.57463086e-02 -4.68251497e-01 -3.68343107e-03
   9.81868029e-01 -4.20358658e-01 -5.45188636e-02 -4.51442212e-01
  -1.83036059e-01  6.77572668e-01  1.16042241e-01  1.54579937e-01
   5.65259874e-01  1.83929086e-01  8.43855739e-01  2.39227444e-01
   6.10969961e-02 -5.18004239e-01  2.15116888e-02 -1.20255813e-01
  -2.10433438e-01 -1.61973611e-01 -9.20219868e-02  1.21949047e-01
   2.34432876e-01  4.56921697e-01 -5.36687613e-01 -1.92415893e-01
  -1.12277269e-02 -8.67999494e-02  4.30712193e-01  3.57330322e-01
  -3.43276978e-01 -1.11858174e-03 -8.62922147e-02  5.68372607e-01
  -1.64694309e-01 -1.02641135e-02 -3.04551452e-01 -1.05233952e-01
   2.31941938e-02 -2.17774689e-01  3.49239886e-01 -3.13810766e-01
   5.26237845e-01  2.65240937e-01 -5.27296551e-02  3.07237089e-01
   1.64248466e-01  1.03462920e-01 -3.80344182e-01 -2.42949605e-01
   4.21214402e-01  2.30814770e-01  2.30781645e-01  4.21311229e-01
  -2.37741187e-01 -9.00204852e-02  3.10524285e-01  3.39663684e-01
   1.98459446e-01  4.18657213e-01 -3.90342951e-01 -4.36680913e-02
  -3.24163973e-01  2.99055099e-01  2.73938030e-01 -2.90868506e-02
  -1.80302292e-01 -6.43003583e-01 -2.68438756e-01  2.58091867e-01
   3.05692077e-01 -2.97341168e-01  8.93231630e-02 -4.40527469e-01
  -7.73056984e-01 -1.60014391e-01  1.38532788e-01 -7.38536716e-01
  -1.64296344e-01  2.08169609e-01 -1.02393746e-01  2.77774632e-01
  -2.82936752e-01  1.10135272e-01 -5.59751153e-01  1.60747916e-01
  -1.12456307e-01  3.35685879e-01  1.64862156e-01 -2.78661519e-01
   1.60603970e-01  4.13544208e-01 -3.07950348e-01 -2.72474438e-02
   2.01274157e-02 -1.62624180e-01  5.85616007e-02 -9.40173864e-01
   5.62069193e-02 -1.41005829e-01 -2.34296948e-01  1.66323796e-01
   4.23554212e-01  9.86377895e-02  6.84781596e-02 -6.50556803e-01
   3.87966424e-01  6.63644850e-01  1.50800273e-01 -4.74523127e-01
   3.50770473e-01  3.93995285e-01  3.28933209e-01  2.04515636e-01
  -2.07581952e-01 -5.61850905e-01 -7.64838904e-02 -5.98778427e-01
   1.15272731e-01  4.90354151e-02  1.32903308e-02  2.07780793e-01
  -8.77514184e-02 -7.43612945e-02  9.14138258e-02 -6.23585522e-01
  -1.26642585e-02 -6.32941544e-01  1.31220251e-01  2.03464866e-01
  -6.43662989e-01  5.36251545e-01  1.46728650e-01 -3.08047146e-01
   1.38973296e-01  6.79948926e-02  2.00682119e-01 -5.87242603e-01
  -7.29923844e-02  2.59301692e-01  5.40646762e-02 -1.98613092e-01
   2.01048762e-01  7.88938403e-02  6.07666373e-01 -4.46455508e-01
   4.00543213e-01  4.85505760e-01  1.33734211e-01 -2.52500087e-01
   5.12345135e-01 -3.41068476e-01  2.50263065e-01 -5.16757369e-01
   4.22116220e-01 -3.64944756e-01  5.96369505e-02  2.85456777e-01
   2.41447240e-02  1.35792807e-01 -4.15638804e-01  6.36375070e-01
  -4.47245032e-01  2.26150334e-01 -3.55103761e-01  1.23095557e-01
  -4.27167639e-02 -4.31500494e-01 -6.31664693e-03 -3.46273869e-01
  -5.87692857e-01 -1.28783599e-01 -2.91683912e-01 -6.05832934e-02
   4.56360847e-01 -4.53914762e-01 -4.73422945e-01 -2.23444462e-01
   2.31774971e-02  2.23046094e-01  2.97252834e-02 -4.83233109e-02
   2.37387612e-01  3.83375168e-01 -2.50337660e-01 -3.80715430e-02
  -5.02874434e-01  3.43140364e-01  1.92654610e-01 -4.70892161e-01
   2.04200178e-01 -6.73158765e-02 -3.64311576e-01  2.58879274e-01
  -2.70090818e-01 -5.19712210e-01 -6.34175777e-01 -2.54016399e-01
  -3.21124256e-01 -2.06200510e-01  2.73228019e-01  1.74755603e-01
  -2.16847152e-01 -2.72930600e-02 -2.51526892e-01 -5.64154267e-01
  -1.28555104e-01  3.00920486e-01  3.42200041e-01 -1.32454112e-02
   5.82739472e-01  2.17595696e-03  1.03236631e-01  2.58115351e-01
  -1.77036136e-01 -9.36193317e-02 -1.35126024e-01  4.46459472e-01
  -2.02759877e-01 -4.21825051e-03  2.14683443e-01 -5.09267807e-01
  -1.51146978e-01  2.13405684e-01 -3.45479399e-02  4.80436645e-02
  -1.33089453e-01 -3.31834257e-01 -3.56496692e-01  3.91432643e-01
   7.35418200e-02  2.35787600e-01 -2.24946618e-01  5.03463805e-01
  -6.43615425e-01  1.40702218e-01  4.04712558e-03  2.37814754e-01
  -7.09427297e-01 -3.08517903e-01 -1.30305931e-01  1.58109382e-01
   3.37559760e-01  5.46296760e-02  2.27313042e-02 -9.16732550e-02
   4.43266928e-01  7.75632262e-03  2.47467011e-02  2.61645794e-01
  -2.92593271e-01 -2.29857519e-01 -4.49732244e-01 -4.43669975e-01
   1.49660856e-02  2.97534466e-02 -2.22745657e-01  1.35632515e-01
  -8.47346783e-02 -7.50894845e-02 -1.28094167e-01  2.43405718e-02
   7.61230364e-02 -1.22797400e-01  1.80078059e-01 -1.68902874e-01
  -3.90722901e-01 -7.63324738e-01 -5.55660427e-01  8.44415128e-02
  -3.56801003e-01  3.34083974e-01  1.19065307e-01 -2.23231867e-01
   2.39179403e-01  4.10869122e-02 -1.05145946e-01 -8.41660351e-02
  -1.25603750e-02 -9.09478217e-02  3.89721245e-01  3.37291360e-01
  -3.51415515e-01  1.95735067e-01  5.22657037e-02  3.45999479e-01
   2.66945034e-01  6.36934042e-02 -4.56354320e-01  1.07113101e-01
  -4.10517633e-01 -1.48972064e-01  4.39400911e-01 -4.59120512e-01
  -1.46285623e-01 -3.12922537e-01 -1.58979312e-01  3.34101439e-01
  -3.89385939e-01  2.42836863e-01  5.95577538e-01 -1.38830423e-01
  -1.39441788e-01 -1.95736110e-01  3.87013406e-02 -3.48073810e-01
  -3.81339222e-01 -1.04521684e-01 -5.13182357e-02  3.16595972e-01
   4.31497037e-01  1.87604144e-01 -4.67181623e-01 -1.62349761e-01
  -1.29547209e-01 -8.28453302e-01  1.94661781e-01 -3.93279612e-01
  -5.72484553e-01 -4.17560816e-01  4.46977794e-01 -6.71254992e-02
   5.24590433e-01 -1.69823423e-01  4.04853225e-02  4.11147714e-01
  -3.48503023e-01 -7.71294236e-02  4.78017926e-01 -1.63011953e-01
   1.21190980e-01  1.54257268e-02  4.31800783e-01  9.40551534e-02
   6.22319058e-03  1.11667834e-01  4.64407355e-01  1.32269442e-01
   7.56642967e-02  6.40671179e-02  3.89759839e-01 -3.60094190e-01
  -3.41098368e-01 -3.00048292e-01  3.69287670e-01 -4.44999263e-02
  -9.19769481e-02  6.07244372e-02  5.62664121e-02  4.69509900e-01
  -3.04481298e-01  9.70490351e-02  3.27563062e-02 -4.25353587e-01
   7.76529908e-02  7.85766095e-02  3.84713352e-01  3.05059880e-01
  -2.41286844e-01 -1.09008238e-01  3.16573024e-01  7.06358850e-01
   3.43150422e-02  2.54470706e-01  7.20705330e-01  4.90306050e-01
   5.42692363e-01 -5.86065054e-02  1.54356733e-01 -4.42238003e-02
   2.35446990e-02 -9.77447391e-01  2.06657171e-01 -5.13286293e-01]], shape=(1, 512), dtype=float32)
openvino [[ 0.51548785 -0.64077765 -0.37209186 -1.271211   -0.5786855   0.01113631
   0.34689552  0.42296135  0.8220525   0.5639597   0.96790946 -0.22792497
  -1.3933531  -0.9898531  -0.58172035 -0.54126537 -1.3497146  -0.41673923
  -0.14833996 -0.05782678  0.16396469 -0.9092252   0.71382564  1.2995716
   0.63787705  1.2088499   0.2744637  -1.3904257  -1.5724828   0.22422503
  -0.7049289  -0.21844886  0.39152747  0.22035593  0.19728407  1.017784
  -0.27378875  0.01115757 -0.16365887 -0.1024081   1.6310158  -0.21915591
  -0.04037778 -1.0066544  -1.0878952  -0.44697464 -0.25355297 -1.0102471
   0.47054178 -0.7581081  -0.06055709  0.25037116 -1.2228707   0.3670869
  -0.06738102 -0.46822375  1.5119437   0.2723548  -0.61996543  0.3165001
  -0.1482751  -0.43913725  0.845614   -0.25069094  1.2530974  -0.6328243
   1.1196929   0.5411788  -0.28524008  0.0475758   0.18718281  0.24560544
  -0.9478284  -0.41731292 -0.45503947  0.35481727 -0.19155854 -0.7865696
  -0.32172766 -1.0783545  -0.42513978  0.37673908 -0.38967472 -0.23553535
   0.9986328  -0.99737495 -0.9649888  -1.0331044   0.11210491 -0.28353733
   0.05876181 -0.29643247  0.09124547  0.3220855   1.0513767   0.36319262
  -0.6881083  -0.72035855  0.34271696  0.06891885  1.1334171   0.04671713
   1.0406866   0.11264829 -0.34719336 -0.277419   -0.32087356  0.5217163
  -1.0323286   0.51361394 -0.695282    0.344843    0.3562817  -0.08276511
  -0.13762595  0.02632765 -0.34684157 -0.07078539  0.21781445  0.4353515
  -0.926854    1.1984277   0.18413308 -0.45016408 -0.01947947 -0.5458225
  -0.2752342   0.08767112 -0.5033858   0.5389004  -0.13104293  0.3670676
  -0.3189904   0.48550922 -0.28703508 -0.4686077  -0.3890769  -0.92605275
   0.01940467 -0.49091265 -0.36175698  1.2900174   0.26911354  0.9193928
   0.43561575 -0.5127272  -0.458336    0.02177384  0.5803926   1.1821681
   0.48884517  0.5655931   0.33676472 -0.6521775   1.465999    0.17713614
  -0.40815765 -0.45189953  0.24637482 -0.3305205  -0.49634337  0.47012556
  -0.46531445  0.29759705 -0.57661724 -0.5767486  -0.1397522   0.0813358
  -0.30036962  0.3581287  -0.68305886  0.04855412  0.05143647 -0.40173456
  -0.39132035  0.6456503   0.2216126   0.29592055 -1.4758294   0.93563294
   0.8539618  -0.46945468  0.8209374  -0.8914087   0.03082645  0.1462229
  -0.57618546 -0.2759489   0.8186034   0.8581794  -0.38999856  0.11854231
   0.45345032  0.15379518 -1.1884941  -0.5156116   0.05637799 -0.9038477
   0.3434903   0.51654124  0.30770904  0.8982548  -0.52278656  0.8871832
  -0.01276591  0.5061512   0.5036938  -0.0821838   0.09615165 -0.8344743
  -0.14011118  0.03273579 -0.9055573   0.86598176  0.14748317 -0.7162478
   1.0571878   0.7089901  -0.815809   -0.4448752  -0.12323277 -0.9245919
  -0.34160307 -0.4214657   0.38639474  0.14993477  0.5584537  -0.28592774
   1.0504985   0.97713685 -0.5708088   0.59806585 -1.5962276   0.11751665
   1.0977567   0.43338245 -1.108989   -0.23130676 -0.9534019  -0.2457592
  -0.37176546  0.2642584  -0.11904491  0.06817093  1.0792352   0.7652346
  -0.88052446  0.34236312  0.9924549   0.8852154   0.92212963 -0.54880404
  -0.71361333  0.5222159   0.4941603  -0.7514542   0.7003276   0.12363701
  -0.22582524  1.6871998  -0.7533804  -1.2352988  -0.08191957  0.76844496
  -0.72859603 -0.6094127   0.2627983  -0.20436125 -0.6147939  -0.35057956
   0.0271228   0.3862368   0.5932603   0.10286173 -0.69023466  0.26412565
  -0.46874177  0.26779947 -0.45582032 -0.01135573 -0.11196291 -0.14853342
   0.47218454  0.5906627  -0.2106334  -0.33965617 -0.76985955  0.80323476
   0.82721436 -0.09654519  0.10872248  0.03656422  0.5261409   0.2102852
   0.20756012 -0.21645778  0.5904711  -0.00347569 -0.39232832  0.10466373
   1.6118118   0.3473379  -0.73349845 -0.24914625 -0.84806764  0.25869676
   0.80240244  0.7365031   0.47475877 -0.3544807   1.5092319  -0.22304264
   0.33897206  0.25537497 -0.02301641 -0.5415784   0.66444236  0.3971383
  -0.7806523  -0.8612099  -1.1081955   0.10937576 -0.45262265 -0.7430046
   0.12292787  1.1819729   0.2143211  -0.2256586  -0.5709912  -0.3015349
  -0.52585566  0.3578801  -0.5839492  -0.48946315 -0.6076751  -0.19567722
  -0.6140451   0.7004341   0.38716644  1.6886289  -0.5452353  -0.97996694
  -0.32942596 -0.10875082 -0.6095432   0.2098016   0.53854024  0.65088147
  -0.07200216 -0.88330936  0.18442917  0.08311453  0.3269477  -0.36617213
  -0.6322907   0.08240132 -0.4620544   0.12950587 -0.808972    0.40628397
  -0.6494337   1.1383693  -0.08790914 -0.20412616  0.34104168  0.78643423
  -0.34002733  0.12283219  0.54933155 -0.42447808 -0.6000206  -0.4467917
   1.0473943   0.6439354  -0.00374627  0.48257148 -0.8132808  -0.26481965
   0.16337113  0.15892692 -0.6065468  -0.7153446   0.43337542 -0.6062828
  -0.37778887 -0.09532332  0.00487174  0.84458303  0.5486102  -0.7750757
   0.5370531  -0.48984832  0.35426915 -1.1059593   0.62400305  1.1648893
   0.50287163  0.4774003   0.42773694 -0.10555767 -0.34728077  0.6840893
   0.98837584 -0.1832315   1.2151161  -0.22331065 -0.47117877  0.22956565
  -0.44763082 -0.09954199 -0.9361087   0.62853014 -0.30793417 -0.3331843
  -0.3483532   0.6684719  -1.428018   -0.06143522 -0.37524688 -0.3527184
   0.13498259  1.1027684  -0.41352507 -0.1320957   0.5166499   0.4890137
   1.0368373  -0.08403514 -0.49701387 -0.6566775   0.752736   -0.07386912
  -0.6212046  -1.2996448  -0.60690814  0.04154817 -0.7098363   0.43613976
   0.7857406  -0.3516778  -0.64339656  0.14856672  0.22253625 -0.6759862
  -0.7276374  -0.7008185   0.6353034  -0.35727358  0.5461697  -0.48591155
   0.5048038  -0.05233821 -0.43766984  1.0890671  -0.58611155 -0.91183114
   0.24011545 -1.1912694  -0.26881397 -0.4706458  -0.73460466 -0.44907594
   0.53204143  0.3818817   0.75427866  0.53064716  0.1212761  -0.0882329
   0.9477394  -0.52271676  0.6268884  -0.5091083  -0.38708448 -0.20498653
   1.6610403   1.0720476  -0.39468333  1.5845281   0.5384499  -1.0613152
   0.34429428  0.9565498   0.15889402 -0.4621487  -1.5343666   0.25490907
   0.82639337  0.69635665 -0.07266285  0.10611603 -0.53859115  0.08017552
   0.88136125 -1.2797347   1.2626778  -0.155864   -0.1508013   0.5680413
  -0.74131143  0.40690756  0.36419323  0.2950086  -0.27142894  0.10383748
  -0.7087336   0.41043612  0.3215009   0.5453005   0.0656197   0.52971995
   0.0706569   0.41921788]]
tflite [[ 5.99302411e-01  1.02025345e-01 -1.38628691e-01 -8.28156173e-02
  -1.15055490e+00 -3.06971192e-01  5.69539487e-01  3.97434711e-01
  -2.24763036e-01  9.62565064e-01  3.98769349e-01 -6.01031065e-01
  -3.35068703e-01  5.51891327e-02  2.58903563e-01  8.24933946e-01
  -3.71180713e-01  1.30760789e-01 -6.15386963e-02  3.68146896e-01
  -6.39128327e-01 -5.06524861e-01 -6.92918003e-01 -5.23515642e-02
  -2.34094903e-01  4.19486523e-01 -1.75196305e-01  1.35073781e-01
   1.37423053e-01  3.71939659e-01 -7.57364035e-01 -1.65197760e-01
  -4.83992279e-01  1.56146824e-01  1.08464055e-01  5.09613693e-01
  -2.41916001e-01  6.75232783e-02 -5.54440558e-01 -2.77796894e-01
  -1.58138424e-01 -1.44299418e-01  9.15497839e-02  1.91471785e-01
  -7.41668105e-01  3.42759825e-02 -6.00280166e-01 -2.28537440e-01
  -4.45316643e-01  2.73516148e-01  5.44285215e-02 -2.78870285e-01
   2.71284074e-01 -7.55840600e-01  3.21335018e-01  5.30450046e-01
  -2.14744583e-02  4.46705192e-01 -2.13501781e-01  3.48506093e-01
   1.68628842e-01  1.40363023e-01 -4.20724332e-01  3.39272432e-02
  -1.64799601e-01 -4.67052519e-01 -4.99580741e-01  1.74603969e-01
   2.19653752e-02 -6.15903139e-02 -4.65217978e-03 -9.56437588e-01
   3.40951502e-01  6.66563660e-02 -1.76118463e-01 -1.75438702e-01
   1.61917329e-01  2.77956605e-01 -3.21817487e-01 -2.15372801e-01
   2.91383982e-01 -1.39457613e-01  1.85975045e-01 -3.67198706e-01
   4.58539307e-01  2.26939231e-01  2.88295299e-01 -4.60933410e-02
   3.52050781e-01 -3.29641104e-01  2.04480022e-01  2.30770141e-01
   2.81754464e-01  5.43769300e-01  3.33087325e-01 -2.40768790e-02
  -1.28103867e-01  3.75635624e-01  2.54697591e-01  2.48563856e-01
  -1.57808319e-01  1.24864176e-01  2.64614552e-01  1.92338198e-01
   1.71802312e-01  2.72325397e-01  3.77641678e-01 -7.74843097e-02
  -8.55419517e-01 -3.43676656e-01  9.41789821e-02  1.18282728e-01
   4.13011163e-02  5.20850569e-02 -4.33011651e-01  5.56076050e-01
   5.71102649e-02  2.88955212e-01  3.64323199e-01  1.77273124e-01
   4.31516677e-01 -1.54330581e-01  6.07010782e-01  3.29576373e-01
  -6.40121400e-02 -2.94239908e-01 -2.36447453e-01 -5.47835231e-01
   1.37121290e-01  9.52145755e-02  1.08205721e-01 -6.83996603e-02
  -1.04012400e-01  5.60602129e-01  2.05778807e-01 -3.62144381e-01
  -5.36920913e-02  9.56986099e-02 -4.68330145e-01 -3.70404124e-03
   9.81900871e-01 -4.20493007e-01 -5.44791967e-02 -4.51375991e-01
  -1.82982937e-01  6.77720726e-01  1.15959331e-01  1.54478118e-01
   5.65294743e-01  1.83801472e-01  8.43771815e-01  2.39277601e-01
   6.10556230e-02 -5.17972052e-01  2.16788054e-02 -1.20340481e-01
  -2.10311279e-01 -1.62144676e-01 -9.19791311e-02  1.22033730e-01
   2.34419703e-01  4.56838906e-01 -5.36593318e-01 -1.92544878e-01
  -1.13009661e-02 -8.70669186e-02  4.30729210e-01  3.57337058e-01
  -3.43304843e-01 -1.13294274e-03 -8.62161219e-02  5.68316877e-01
  -1.64954782e-01 -1.03202276e-02 -3.04596215e-01 -1.05231851e-01
   2.32337564e-02 -2.17759192e-01  3.49267006e-01 -3.13758284e-01
   5.26261449e-01  2.65198886e-01 -5.25823608e-02  3.07105243e-01
   1.64154038e-01  1.03411600e-01 -3.80188823e-01 -2.42951542e-01
   4.21311438e-01  2.30796829e-01  2.30954871e-01  4.21259820e-01
  -2.37564638e-01 -9.01707485e-02  3.10363889e-01  3.39536220e-01
   1.98628098e-01  4.18716729e-01 -3.90469611e-01 -4.38272059e-02
  -3.24129999e-01  2.99053997e-01  2.73916215e-01 -2.90430784e-02
  -1.80167675e-01 -6.43005967e-01 -2.68406630e-01  2.58116871e-01
   3.05612326e-01 -2.97299355e-01  8.93085301e-02 -4.40638065e-01
  -7.73047447e-01 -1.60009056e-01  1.38401508e-01 -7.38459945e-01
  -1.64298922e-01  2.07831115e-01 -1.02401257e-01  2.77849197e-01
  -2.82950014e-01  1.10249773e-01 -5.59964240e-01  1.60741836e-01
  -1.12534150e-01  3.35582197e-01  1.64934307e-01 -2.78715163e-01
   1.60627693e-01  4.13482040e-01 -3.07952732e-01 -2.71093100e-02
   2.02490687e-02 -1.62498698e-01  5.88398501e-02 -9.40332949e-01
   5.61766326e-02 -1.41050011e-01 -2.34331772e-01  1.66360036e-01
   4.23553050e-01  9.86243039e-02  6.83097690e-02 -6.50489211e-01
   3.87944341e-01  6.63942158e-01  1.50898784e-01 -4.74413633e-01
   3.50842267e-01  3.94045055e-01  3.29041839e-01  2.04649150e-01
  -2.07588628e-01 -5.61717451e-01 -7.63456672e-02 -5.98916888e-01
   1.15104504e-01  4.90489751e-02  1.34362988e-02  2.07548469e-01
  -8.78730714e-02 -7.43052810e-02  9.15648267e-02 -6.23747706e-01
  -1.25042200e-02 -6.32824242e-01  1.31414175e-01  2.03285515e-01
  -6.43712878e-01  5.36271036e-01  1.46743417e-01 -3.07969332e-01
   1.39250875e-01  6.78908825e-02  2.00795814e-01 -5.87143183e-01
  -7.29386210e-02  2.59103656e-01  5.41479960e-02 -1.98473826e-01
   2.01135278e-01  7.90135413e-02  6.07809782e-01 -4.46634591e-01
   4.00616407e-01  4.85493004e-01  1.33625671e-01 -2.52437323e-01
   5.12398541e-01 -3.41052920e-01  2.50229210e-01 -5.16862154e-01
   4.22099739e-01 -3.64758670e-01  5.95833659e-02  2.85447806e-01
   2.41271704e-02  1.35824293e-01 -4.15771753e-01  6.36272669e-01
  -4.47350919e-01  2.26052552e-01 -3.55051726e-01  1.23334989e-01
  -4.24687788e-02 -4.31559145e-01 -6.38943911e-03 -3.46207321e-01
  -5.87660193e-01 -1.28857478e-01 -2.91680664e-01 -6.04712814e-02
   4.56237227e-01 -4.54046547e-01 -4.73559678e-01 -2.23460525e-01
   2.31597051e-02  2.23081291e-01  2.96940506e-02 -4.84968126e-02
   2.37346962e-01  3.83187175e-01 -2.50171900e-01 -3.80795598e-02
  -5.02867579e-01  3.43029350e-01  1.92597657e-01 -4.70788866e-01
   2.04206392e-01 -6.72628433e-02 -3.64420950e-01  2.58931041e-01
  -2.70123184e-01 -5.19733548e-01 -6.34155035e-01 -2.53973424e-01
  -3.21116954e-01 -2.06074655e-01  2.73086131e-01  1.74708098e-01
  -2.16826677e-01 -2.72150077e-02 -2.51614124e-01 -5.64132929e-01
  -1.28697589e-01  3.00810546e-01  3.42334449e-01 -1.33354217e-02
   5.82770586e-01  2.15938687e-03  1.03342332e-01  2.58213758e-01
  -1.77153364e-01 -9.36340243e-02 -1.34907484e-01  4.46408033e-01
  -2.02900440e-01 -4.14645672e-03  2.14792818e-01 -5.09220779e-01
  -1.51245326e-01  2.13496476e-01 -3.45522985e-02  4.80326004e-02
  -1.33083105e-01 -3.31801742e-01 -3.56566995e-01  3.91508102e-01
   7.36040547e-02  2.35680133e-01 -2.25060314e-01  5.03517389e-01
  -6.43589258e-01  1.40562505e-01  4.00823355e-03  2.37804592e-01
  -7.09515333e-01 -3.08364749e-01 -1.30239353e-01  1.58263281e-01
   3.37535828e-01  5.46374172e-02  2.27647200e-02 -9.16529819e-02
   4.43249941e-01  7.77623057e-03  2.46850401e-02  2.61602461e-01
  -2.92746544e-01 -2.29798123e-01 -4.49536264e-01 -4.43617642e-01
   1.49222240e-02  2.98450440e-02 -2.22584397e-01  1.35466024e-01
  -8.44657719e-02 -7.51532167e-02 -1.28077328e-01  2.42796838e-02
   7.62560591e-02 -1.22968137e-01  1.79896593e-01 -1.68862894e-01
  -3.90759796e-01 -7.63460219e-01 -5.55712700e-01  8.45817029e-02
  -3.56734991e-01  3.33852261e-01  1.19136050e-01 -2.23146066e-01
   2.38984838e-01  4.11878377e-02 -1.05135560e-01 -8.41244310e-02
  -1.24813244e-02 -9.09136832e-02  3.89571905e-01  3.37341905e-01
  -3.51366013e-01  1.95728391e-01  5.21202609e-02  3.45972151e-01
   2.66834289e-01  6.35720193e-02 -4.56418216e-01  1.07137896e-01
  -4.10526514e-01 -1.49002954e-01  4.39391553e-01 -4.59108591e-01
  -1.46406740e-01 -3.12810481e-01 -1.58830941e-01  3.34200352e-01
  -3.89441431e-01  2.42814928e-01  5.95506787e-01 -1.38821959e-01
  -1.39385551e-01 -1.95567906e-01  3.86707038e-02 -3.48027915e-01
  -3.81326288e-01 -1.04506627e-01 -5.11747263e-02  3.16478431e-01
   4.31535542e-01  1.87419966e-01 -4.67267454e-01 -1.62407160e-01
  -1.29435092e-01 -8.28787923e-01  1.94753394e-01 -3.93228710e-01
  -5.72491944e-01 -4.17554617e-01  4.47066277e-01 -6.71352893e-02
   5.24409652e-01 -1.69600561e-01  4.06057090e-02  4.11079764e-01
  -3.48637879e-01 -7.72973150e-02  4.78040397e-01 -1.62848786e-01
   1.21055201e-01  1.52831823e-02  4.31878507e-01  9.41661522e-02
   6.30949065e-03  1.11588761e-01  4.64250177e-01  1.32279783e-01
   7.58552998e-02  6.41965494e-02  3.90068769e-01 -3.60179007e-01
  -3.41287583e-01 -3.00062537e-01  3.69384766e-01 -4.45377827e-02
  -9.19121057e-02  6.08072653e-02  5.61715886e-02  4.69482839e-01
  -3.04593325e-01  9.69407856e-02  3.28494459e-02 -4.25284564e-01
   7.76131004e-02  7.85150528e-02  3.84676933e-01  3.05178434e-01
  -2.41470277e-01 -1.09021798e-01  3.16628456e-01  7.06635952e-01
   3.42841074e-02  2.54443109e-01  7.20716178e-01  4.90371257e-01
   5.42798936e-01 -5.84390834e-02  1.54369354e-01 -4.42765802e-02
   2.35848576e-02 -9.77475226e-01  2.06788510e-01 -5.13243794e-01]]
onnx [array([[ 0.5154878 , -0.64077723, -0.37209094, -1.2712114 , -0.5786855 ,
         0.01113635,  0.34689307,  0.4229616 ,  0.82205373,  0.56396127,
         0.96790886, -0.22792593, -1.3933529 , -0.98985237, -0.5817212 ,
        -0.54126537, -1.3497163 , -0.41674024, -0.14834106, -0.05782611,
         0.16396402, -0.90922594,  0.71382433,  1.2995716 ,  0.63787764,
         1.2088503 ,  0.27446502, -1.3904245 , -1.5724823 ,  0.22422653,
        -0.7049296 , -0.21844684,  0.391527  ,  0.2203559 ,  0.19728403,
         1.0177824 , -0.27378738,  0.01115812, -0.16365972, -0.10240981,
         1.6310164 , -0.21915475, -0.04037616, -1.0066563 , -1.0878955 ,
        -0.4469741 , -0.2535531 , -1.0102484 ,  0.47054267, -0.75810856,
        -0.06055658,  0.2503714 , -1.2228713 ,  0.36709034, -0.06738205,
        -0.46822408,  1.5119438 ,  0.27235442, -0.61996615,  0.31649947,
        -0.14827454, -0.43913758,  0.8456128 , -0.2506912 ,  1.2530975 ,
        -0.63282347,  1.119692  ,  0.5411777 , -0.28524172,  0.04757464,
         0.18718201,  0.24560611, -0.94782907, -0.41731292, -0.4550401 ,
         0.35481837, -0.19155857, -0.7865707 , -0.32172912, -1.0783548 ,
        -0.42514017,  0.37674087, -0.38967732, -0.23553404,  0.99863404,
        -0.9973753 , -0.96499085, -1.0331024 ,  0.11210354, -0.28353596,
         0.05876311, -0.29643634,  0.0912458 ,  0.3220856 ,  1.0513757 ,
         0.36319366, -0.68810755, -0.72035944,  0.34271547,  0.06891936,
         1.1334162 ,  0.04671521,  1.0406873 ,  0.11264773, -0.3471917 ,
        -0.2774206 , -0.32087374,  0.5217142 , -1.0323291 ,  0.5136141 ,
        -0.69528365,  0.34484288,  0.35628206, -0.08276408, -0.13762596,
         0.02632755, -0.34684226, -0.07078622,  0.21781507,  0.43534997,
        -0.9268535 ,  1.1984282 ,  0.18413284, -0.45016336, -0.01947778,
        -0.54582405, -0.2752366 ,  0.08767001, -0.5033858 ,  0.53890157,
        -0.13104282,  0.3670682 , -0.318991  ,  0.48551148, -0.28703555,
        -0.46860933, -0.38907525, -0.92605203,  0.01940417, -0.49090824,
        -0.3617588 ,  1.2900164 ,  0.26911288,  0.9193914 ,  0.4356175 ,
        -0.51272726, -0.45833722,  0.02177086,  0.58039296,  1.1821656 ,
         0.4888469 ,  0.5655913 ,  0.33676434, -0.6521786 ,  1.4659988 ,
         0.17713633, -0.40815836, -0.45189995,  0.24637577, -0.3305214 ,
        -0.4963452 ,  0.47012448, -0.46531314,  0.29759523, -0.5766169 ,
        -0.5767493 , -0.13974956,  0.08133373, -0.300369  ,  0.35812905,
        -0.68305874,  0.04855425,  0.05143687, -0.40173566, -0.39131895,
         0.64564794,  0.22161499,  0.29591838, -1.4758275 ,  0.9356326 ,
         0.85396355, -0.469453  ,  0.8209389 , -0.8914095 ,  0.03082947,
         0.14622149, -0.576184  , -0.2759486 ,  0.8186026 ,  0.85818046,
        -0.38999957,  0.11854243,  0.45344976,  0.15379517, -1.188492  ,
        -0.51561236,  0.05637882, -0.9038482 ,  0.34348974,  0.5165408 ,
         0.3077111 ,  0.8982531 , -0.5227878 ,  0.88718206, -0.01276727,
         0.5061501 ,  0.50369334, -0.08218326,  0.09615223, -0.8344729 ,
        -0.14011312,  0.03273518, -0.90555775,  0.8659818 ,  0.14748284,
        -0.71624887,  1.0571877 ,  0.70898914, -0.81580913, -0.4448742 ,
        -0.12323324, -0.924591  , -0.34160322, -0.42146394,  0.3863954 ,
         0.14993328,  0.5584539 , -0.2859294 ,  1.0505018 ,  0.9771355 ,
        -0.57080996,  0.59806454, -1.596229  ,  0.11751863,  1.0977573 ,
         0.43338406, -1.1089923 , -0.23130755, -0.9534019 , -0.24575844,
        -0.371765  ,  0.26425678, -0.1190471 ,  0.06817108,  1.0792334 ,
         0.7652348 , -0.8805243 ,  0.34236085,  0.9924568 ,  0.8852171 ,
         0.9221285 , -0.5488056 , -0.7136135 ,  0.52221876,  0.49415964,
        -0.75145173,  0.70032763,  0.12363697, -0.2258255 ,  1.6871992 ,
        -0.7533797 , -1.2352982 , -0.08191754,  0.76844525, -0.72859603,
        -0.60941166,  0.26279667, -0.20435917, -0.614794  , -0.35057953,
         0.02712208,  0.38623813,  0.5932603 ,  0.10286272, -0.6902336 ,
         0.2641254 , -0.4687415 ,  0.26779842, -0.45582038, -0.0113575 ,
        -0.11196309, -0.14853457,  0.4721838 ,  0.59066415, -0.21063417,
        -0.33965668, -0.7698602 ,  0.8032356 ,  0.82721454, -0.09654506,
         0.10872323,  0.03656298,  0.52614063,  0.21028775,  0.20755973,
        -0.21645768,  0.5904697 , -0.00347621, -0.3923267 ,  0.10466336,
         1.611814  ,  0.3473361 , -0.73349714, -0.24914531, -0.8480675 ,
         0.25869703,  0.8024009 ,  0.7365004 ,  0.47476125, -0.3544828 ,
         1.509232  , -0.22304434,  0.3389738 ,  0.2553767 , -0.0230158 ,
        -0.54157954,  0.66444105,  0.39713785, -0.7806512 , -0.86120796,
        -1.1081952 ,  0.10937573, -0.45262137, -0.74300516,  0.12292777,
         1.1819724 ,  0.21431866, -0.22565797, -0.57099205, -0.30153412,
        -0.52585685,  0.35787824, -0.5839484 , -0.48946372, -0.6076752 ,
        -0.19567825, -0.61404514,  0.70043385,  0.38716692,  1.6886294 ,
        -0.5452343 , -0.9799677 , -0.3294258 , -0.10875139, -0.60954165,
         0.20980231,  0.53854024,  0.6508825 , -0.07200328, -0.8833107 ,
         0.18442954,  0.08311626,  0.326949  , -0.36617127, -0.6322913 ,
         0.08239983, -0.46205467,  0.1295066 , -0.8089716 ,  0.4062853 ,
        -0.6494332 ,  1.1383684 , -0.08790681, -0.20412542,  0.3410409 ,
         0.7864341 , -0.3400272 ,  0.12283432,  0.5493306 , -0.42447776,
        -0.600021  , -0.44679266,  1.0473928 ,  0.6439354 , -0.00374818,
         0.48257074, -0.81327945, -0.26482013,  0.16337223,  0.15892866,
        -0.6065459 , -0.7153439 ,  0.43337408, -0.6062839 , -0.37778884,
        -0.0953238 ,  0.0048714 ,  0.844583  ,  0.5486113 , -0.77507395,
         0.5370525 , -0.48984745,  0.35426962, -1.1059608 ,  0.62400186,
         1.1648906 ,  0.50287175,  0.47740048,  0.42773557, -0.10555652,
        -0.34728056,  0.6840887 ,  0.9883745 , -0.18323015,  1.2151173 ,
        -0.22330944, -0.4711811 ,  0.22956498, -0.44763103, -0.09954342,
        -0.936111  ,  0.6285281 , -0.3079334 , -0.33318508, -0.34835356,
         0.66847265, -1.4280174 , -0.06143822, -0.37524766, -0.35271746,
         0.1349847 ,  1.1027697 , -0.41352326, -0.13209592,  0.51665145,
         0.4890145 ,  1.0368366 , -0.08403666, -0.49701422, -0.6566766 ,
         0.752734  , -0.07386674, -0.6212061 , -1.2996432 , -0.6069102 ,
         0.0415465 , -0.70983666,  0.4361407 ,  0.78574216, -0.35167822,
        -0.64339584,  0.14856678,  0.22253592, -0.6759869 , -0.72763926,
        -0.7008172 ,  0.6353056 , -0.35727325,  0.5461694 , -0.48590982,
         0.50480306, -0.05233669, -0.43766883,  1.0890665 , -0.58611137,
        -0.91183186,  0.24011372, -1.1912706 , -0.26881433, -0.47064418,
        -0.7346033 , -0.44907942,  0.5320411 ,  0.3818806 ,  0.75427735,
         0.53064597,  0.12127674, -0.08823428,  0.94774055, -0.5227172 ,
         0.62689066, -0.5091099 , -0.38708556, -0.20498876,  1.6610429 ,
         1.0720496 , -0.3946836 ,  1.5845271 ,  0.5384481 , -1.0613151 ,
         0.34429345,  0.9565507 ,  0.15889311, -0.46214783, -1.5343676 ,
         0.25490844,  0.826393  ,  0.6963557 , -0.07266048,  0.1061151 ,
        -0.53859234,  0.08017762,  0.8813593 , -1.2797359 ,  1.2626772 ,
        -0.15586475, -0.15079954,  0.56804216, -0.7413099 ,  0.40690494,
         0.3641929 ,  0.29500818, -0.27142698,  0.10383503, -0.70873296,
         0.41043678,  0.3215011 ,  0.54530066,  0.0656192 ,  0.52971804,
         0.07065711,  0.41921505]], dtype=float32)]

Process finished with exit code 0

Source code for simple inference testing code

import scipy.io
import matplotlib.pyplot as plt
import cv2
import numpy as np

import tensorflow as tf

from openvino.inference_engine import IECore, Blob, TensorDesc
import tflite_runtime.interpreter as tflite

image = plt.imread("/home/yzy/Documents/dataset/FaceData/lfw/Wu_Peng/Wu_Peng_0001.jpg")
image = cv2.resize(image, (112, 112))
image_float = np.array((image.astype(np.float32) - 127.5) / 128.0)

ie_core_handler = IECore()
network = ie_core_handler.read_network(model="/home/yzy/PycharmProjects/FaceRecDemo/test_data/iresnet50_relu/iresnet50_relu_sim.xml",
                                       weights="/home/yzy/PycharmProjects/FaceRecDemo/test_data/iresnet50_relu/iresnet50_relu_sim.bin")
executable_network = ie_core_handler.load_network(network, device_name='CPU', num_requests=1)
inference_request = executable_network.requests[0]
tensor_description = TensorDesc(precision="FP32", dims=(1, 3, 112, 112), layout='NCHW')
input_blob = Blob(tensor_description, image_float[None, ...].transpose((0, 3, 1, 2)))
inference_request.set_blob(blob_name="input.1", blob=input_blob)
inference_request.infer()
output_blob_name = next(iter(inference_request.output_blobs))
output = inference_request.output_blobs[output_blob_name].buffer
print(output)

model_TF = tf.saved_model.load("/home/yzy/PycharmProjects/FaceRecDemo/test_data/iresnet50_relu/")
print(model_TF(image_float[None, ...]))

from facerec.facerec import FaceRecognizer
import onnxruntime

model_tf = tflite.Interpreter("ires50_arcface/model_float32.tflite") # FaceRecognizer("ires50_arcface/model_float32.tflite", edgetpu=False)
model_tf.allocate_tensors()
input_details = model_tf.get_input_details()[0]['index']
model_tf.set_tensor(input_details, image_float[None, ...])
output_details = model_tf.get_output_details()[0]['index']
model_tf.invoke()
embedding_tf = model_tf.get_tensor(output_details)

model_onnx = onnxruntime.InferenceSession('/home/yzy/PycharmProjects/FaceRecDemo/test_data/iresnet50_relu/iresnet50_relu_sim.onnx', None)

# embedding_tf = model_tf.inference(image)
embedding_onnx = model_onnx.run([], {'input.1': image_float[None, ...].transpose((0, 3, 1, 2))})

print(embedding_tf)
print(embedding_onnx)
PINTO0309 commented 2 years ago

It is not a problem with the tool, but a lack of dimensional conversion procedures. image

zye1996 commented 2 years ago

It is not a problem with the tool, but a lack of dimensional conversion procedures. image

Is there a workaround to this problem? Or what do you suggest when using the tool for this model? Many thanks!

PINTO0309 commented 2 years ago

https://github.com/PINTO0309/openvino2tensorflow#6-7-replace-weights-or-constant-values-in-const-op-and-add-transpose-or-reshape-or-cast-or-squeeze-or-unsqueeze-or-add-or-multiply-just-beforeafter-the-operation-specified-by-layer_id