intel / ros2_openvino_toolkit

Apache License 2.0
165 stars 84 forks source link

LLM usecase support #314

Open gitpqLee opened 4 months ago

gitpqLee commented 4 months ago

We need a LLM feature integrated into this project:

This repo should be added new code and logic to fulfill this Input->Inference->Output process: 1) Subscribe ROS topic of related audio common message. 2) Adopt LLM agent for the inference body. 3) Publish the corresponding topic(s).

See below picture as the pipeline: image TTS model supported by openvino can refer to: text_to_speech_demo.py LLM like Llama support can leverage OpenVINO-GenAI, please refer to OpenVINO™ GenAI Library

Development Scope:

You have the freedom to choose the intelligent agent.