xmos / fwk_voice

Voice Framework
Other
11 stars 19 forks source link

Integrate VNR model #248

Closed mbanth closed 2 years ago

mbanth commented 2 years ago

Add the VNR model into its destination repository. (AV2b)

shuchitak commented 2 years ago

@andrewdewhurst, could you explain this a bit more. Currently the tensorflow model is in py_vnr repo. The corresponding tflite model and the xcore optimised version of it is in the sw_avona repo as part of the sw_avona lib_vnr module. This makes the most sense to me. Could you let me know, which, if any of the models you want to move to xcore sdk repo? Thanks.

mbanth commented 2 years ago

We will discuss this issue at the 27 April 2022 Voice Pipeline Performance meeting to gain clarity about its scope and necessity.

shuchitak commented 2 years ago

As discussed in the 27 April 2022 Voice Pipeline Performance, py_vnr repo will contain the tf and tflite models, documentation regarding training, tf -> tflite conversion etc. accessible to anyone within XMOS.

mbanth commented 2 years ago

@andrewdewhurst, is this issue still unresolved in your mind?

mbanth commented 2 years ago

@shuchitak, can you please provide information about the state of the documentation in this repository that covers the use of the xformer?

shuchitak commented 2 years ago

By way of documentation, we currently have this python script to which you provide a tflite model as argument and it will convert it to tflite micro using the xformer, generate the tensor arena scratch size needed and model quant spec related defines and optionally copy it to the VNR module so it can run with the new model.

I tried documenting this but it makes much more sense to have this in a script. What's pending is:

shuchitak commented 2 years ago

VNR model is integrated to lib_vnr and documented. All changes merged to develop.