Open AlmogDavid opened 1 year ago
@AlmogDavid Thanks for the query. @quic-akinlawo can you help respond to this? Thanks
Hello @AlmogDavid, your quantsim configuration seems to indicate the opposite - it looks like you are using 16-bit weights if you have default_param_bw = 16?
There is no special 16-bit specific flow in either case. The following DSP tutorial should contain all the info you need: https://developer.qualcomm.com/sites/default/files/docs/snpe/tutorial_inceptionv3.html.
Thanks, i had a typo in my post here. Actually in my code I'm working on 16 bit output and 8 bit weights. I also opened an issue in SNPE, it looks like the DSP does not work in this configuration,
@quic-akinlawo , Anything?
@AlmogDavid From an AIMET point of view, the 16 bit activation and 8 bit weight configuration is supported. If you are seeing accuracy degradation then it could be a specific issue relating to your model running on SNPE DSP. It seems you have already raised an issue with the SNPE DSP team, please follow up with them as they would be best placed to provide assistance/debugging tips.
Hi, I'm trying to train a QAT model using 16 bit activations and 8 bit weights in order to run it on DSP on Snapdragon 888. The training works as it should and the model is converging. When running the model on the Snapdragon CPU (after conversion to DLC + Quantize using SNPE SKD) I'm able to see valid results. When moving to run on DSP I see garbage results... Can you point me to a full tutorial on how to perform the flow I'm interested in? I was not able to find any guide on running 16 bit activation model on top of the DSP. This is the quantsim configuration I'm using:
quantsim = QuantizationSimModel(model=fld_model.to(self.cfg.device), quant_scheme='tf_enhanced', dummy_input=next(iter(image_dl))[data_features.FACE_IMAGE].to(self.cfg.device), rounding_mode='nearest', default_output_bw=8, default_param_bw=16, in_place=False)