AXERA-TECH / ax-samples

Samples code for world class Artificial Intelligence SoCs for computer vision applications.
BSD 3-Clause "New" or "Revised" License
201 stars 41 forks source link

Arcface Model Conversion #139

Open BasharMahasen opened 3 months ago

BasharMahasen commented 3 months ago

Hello @ltdat1201, @shitouren1994 , we are trying to convert Arcface ONNX to joint format to use it on AX620 chip. The conversion for both w600k_R50 & w600k_mbf were successful, however, the cosine on test (pulsar run ...onnx ... joint) is always ZERO. I attached the configurations, and the logs for one trials to narrow it down and solve the issue. I appreicate if you could:

  1. Advise if prototxt file is correct for Arcface mbf.?
  2. As output_config shows, joint model applies permutation on the input (NHWC -> NCHW). does this mean that "pulsar run" of "onnx and joint" can't be done for the same input (to test similarity). For this I manually tried to feed each model a separate npy which matches the input, then manually compute cosine, but still the result is close to zero.
  3. Any suggestions / hints that may help solving this issue?

Attached configs for your reference:

  1. The calibration dataset is float32 arranged in NHWC / RGB as per the onnx model format.

  2. Arcface models: pip install insightface from insightface.app import FaceAnalysis app = FaceAnalysis('buffalo_sc') # downloads w600_mbf.onnx app = FaceAnalysis('buffalo_l') # downloads w600k_r50.onnx

  3. output_config.prototxt.txt

  4. w600k_mbf_float32.prototxt.txt

  5. generate_npyfp32_calibration_dataset.py.txt

  6. pulsar_run_test.log

  7. puslar_build.log