search

Xilinx / Vitis-AI

Vitis AI is Xilinx’s development stack for AI inference on Xilinx hardware platforms, including both edge devices and Alveo cards.
https://www.xilinx.com/ai
Apache License 2.0
1.47k stars 630 forks source link

Editing xmodel files #1294

Open tafk7 opened 1 year ago

tafk7 commented 1 year ago

Is there anyway to edit the weights of an xmodel? Alternatively, is there a way to load vai_c_tensorflow2 onto the petalinux running the DPU?

quentonh commented 1 year ago

@Rellek72 your goal is to update the weights dynamically at runtime? I believe that currently the only way to do this would be to swap in a new xmodel file. The weights in the compiled xmodel are not consumable as they have been reordered and optimized for deployment. Consider however that the ONNX Runtime implementation integrates the compiler into the runtime. The implication is that the xcompiler runs on the target when you leverage ONNX Runtime. Tell us more about your use case.

tafk7 commented 1 year ago

My overall goal is to deploy a design to an UltraScale+ SoC using the traditional Vitis AI flow, then be able to make small tweaks to the weights using transfer learning without leaving the SoC.

My thought was that I could use quantized .h5 output from vai_q_tensorflow2 to do a small amount of quantized training on the ARM core of the SoC. I was hoping to then either manually splice in the weights to the xmodel, or re-compile the xmodel locally on the board

I hadn't looked into the ONNX runtime before, that seems a more promising framework for what I'm trying to do, but I may need to get the vai_q_tensorflow2 running on the ARM core of the SoC.

quentonh commented 1 year ago

@Rellek72 I have been a big proponent of the possibility of implementing Federated Learning on our edge devices, or perhaps the notion of on-device "light" re-training of the model. For Edge applications, this has many potential applications and benefits. Longer term I will continue to consider this and try to advocate for solutions.

AlaaMaz commented 7 months ago

@Rellek72 your goal is to update the weights dynamically at runtime? I believe that currently the only way to do this would be to swap in a new xmodel file. The weights in the compiled xmodel are not consumable as they have been reordered and optimized for deployment. Consider however that the ONNX Runtime implementation integrates the compiler into the runtime. The implication is that the xcompiler runs on the target when you leverage ONNX Runtime. Tell us more about your use case.

Not sure if you're still active, but I'm curious to know, if it's not possible to change the weights, is it possible to access them, as in only read the weights of a specefic layer, I'm working on a watermarking application which would requimre to check for a watermark in the weights in the board. thank you