[ASR Microservice] Incompatible parameters with LLM service for ASR Microservice

[zhlsunshine]

This bug was found when validating the example of AudioQnA, the mega service orchestrates ASR and LLM microservices in this example in the pipeline. The example is okay for mega service pipeline in docker because mega service handle the pipeline data transformation, however, these microservices can not work well in Kubernetes due to the incompatible parameters between ASR and LLM microservices. Suggest that the microservices should follow the unification API definition in LLMParamsDoc in Mega service if they want to work with LLM service together.

[Spycsh]

Hi @zhlsunshine , https://github.com/opea-project/GenAIComps/pull/341 is already merged and you can use it directly on k8s :)

[zhlsunshine]

Hi @Spycsh, thanks so much, we would validate it, then close this issue if it's okay!

[zhlsunshine]

Hi @Spycsh and @lkk12014402, I updated the code from main branch of GenAIComps, then I deployed the AudioQnA example in my K8s cluster, however, I still get below error information:

Error pods:

Error for whisper service:

Error for speecht5 service:

Not sure about that there may be necessary sync with OneClick team, @yongfengdu and @lianhao

[lianhao]

@zhlsunshine PR #406 change the in-container user from root to user with uid=1000. This could cause a file permission issue between old and new model data cache. Please go into your model data cache directory in your host, and do a chmod 777 to all the files under your model data cache dir.

[zhlsunshine]

Hi @lianhao, thanks for your replying, go for the help offline.

[zhlsunshine]

It's okay now, let me close this issue.