feat: Audio bot

[schuettc]

Add audio bot deployment with NIMs.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[amanshanbhag]

One thing that stands out immediately here: Instead of path 2.projects/nims-inference/audio-bot/, can we use something like 2.projects/nims-inference/**use-cases/**audio-bot/ please?

[JoeyTPChou]

2 quick feedbacks:

@amanshanbhag We should consider moving usecase to the git root instead of under projects so people can check the usecases first. For example:

usecases/nims-inference/audio-bot

@schuettc in this section https://github.com/schuettc/awsome-inference/tree/audio-bot/2.projects/nims-inference/audio-bot#nim-containers are you able to check if the CONTAINER_PROFILE and NIM_MANIFEST_PROFILE the right env variables? Because base on the NIM doc the env variable name is NIM_MODEL_PROFILE. Also if we are only use 2 GPUs with LLM, can we change the --gpus "device=0,1,2,3" to `--gpus "device=0,1"?

[schuettc]

@schuettc in this section https://github.com/schuettc/awsome-inference/tree/audio-bot/2.projects/nims-inference/audio-bot#nim-containers are you able to check if the CONTAINER_PROFILE and NIM_MANIFEST_PROFILE the right env variables? Because base on the NIM doc the env variable name is NIM_MODEL_PROFILE. Also if we are only use 2 GPUs with LLM, can we change the --gpus "device=0,1,2,3" to `--gpus "device=0,1"?

The CONTAINER_PROFILE name was already corrected.

[amanshanbhag]

@JoeyTPChou That's a good idea! I think we can have 3.use-cases at root (currently, we have 1.infrastructure, 2.projects).

We can then have this under 3.use-cases/nims-inference

[JoeyTPChou]

Just saw we have .github folder are we able to remove it?

[schuettc]

Just saw we have .github folder are we able to remove it?

Sure