search

Xilinx / Vitis-AI

Vitis AI is Xilinx’s development stack for AI inference on Xilinx hardware platforms, including both edge devices and Alveo cards.
https://www.xilinx.com/ai
Apache License 2.0
1.46k stars 630 forks source link

Some general doubts regarding the use of Vitis AI Optimizer and some specific ones about VAI Optimizer Tensorflow2 #1387

Closed jonGuti13 closed 4 days ago

jonGuti13 commented 9 months ago

I've been working for a couple of months with VAI Optimizer and after a careful reading of both Vitis AI Optimizer User Guide (UG1333) (v3.0) and Vitis AI User Guide (UG1414) (v3.5) I have some doubts that I think are interesting to share and I hope the community or AMD-Xilinx employees can solve.

  1. Do you expect fine-grained pruning to be supported by a DPU? If not, does it exist another Xilinx tool where we could take advantage of fine-grained pruning?
  2. How does the tool use the model sensitivity analysis (model.sens)?
  3. Specifically for TF2 VAI Optimizer, when I specify a certain ratio how does the tool choose between all the models that meet the pruning criteria? I imagine that evaluating all of them one by one would be prohibitively costly. Similarly, if a threshold (instead of a ratio) is specified, how does the tool choose the most adecuate model? From my very short experience, when using the ratio (instead of the threshold) as the pruning criteria, much more "Model Analysis Report"s are output, so this fact distracts me even more as I would think that the ratio criteria would imply analysing fewer models.
  4. During the output of "Model Analysis Report" some options appear (max_depth, min_bytes, min_peak_bytes...) What does that mean?
  5. How does the tool select which channels have to be pruned? Does it use L1 norm criteria?
  6. Can the pruning ratio be multiple of 0.05 (and not multiple of 0.1)?
  7. Will you support One-Step Pruning and Once For All Pruning (OFA) for Tensorflow2 in the very near future?
  8. Even though the output of the eval function can be a metric different from the loss, the threshold can only be specified related to the loss. Am I right?
  9. The release notes from Vitis AI 3.5 state that TensorFlow 2 Optimizer includes "Support for setting the number of channels to be a multiple of the specified number after pruning". Does this mean that we have to manually modify the "channel_divisible" field in the corresponding .spec file?
  10. Finally, will you release the v3.5 of the UG1333 or is all the information already contained in UG1414 v3.5? I think I've read somewhere that UG1333 will be included in UG1414 but I am not 100% sure.

Thank you everyone in advance.

yuwang-xlnx commented 8 months ago
  1. DPU has no support plan for fine-grained pruning and there are currently no other tools that can support it.

2 & 3. The sensitivity analysis is used to determine pruning ratio of each layer. For detailed implementation, you can refer to the code: https://github.com/Xilinx/Vitis-AI/blob/master/src/vai_optimizer/tensorflow/tf_nndct/pruning/runner.py#L421

  1. The output is from tf profiler and users do not need to pay attention to it.
  2. Yes, L1 norm criteria is used.
  3. Currently pruning ratio is multiple of 0.1. (You can make some modifications to make it multiple of 0.05 as Optimizer is open sourced)
  4. No support plan for One-Step and OFA in tensorflow2.
  5. Threshold is related to the metric you specified in model analysis phase. It indicates how much variation you allow for this metric after pruning.
  6. There is an argument for this in pruning API: https://github.com/Xilinx/Vitis-AI/blob/master/src/vai_optimizer/tensorflow/tf_nndct/pruning/runner.py#L384
  7. UG1414 contains all content of UG1333. As Optimizer was open sourced in VAI 3.5, UG1333 was deprecated and the content has been merged in UG1414.
jonGuti13 commented 8 months ago

Hi @yuwang-xlnx,

Thanks for the response and the provided references. I will have a look at them as soon as possible.