search

openvinotoolkit / openvino.genai

Run Generative AI models using native OpenVINO C++ API
Apache License 2.0
76 stars 115 forks source link

[Discussion][Good First Issue]: Verify different LLMs work with text_generation #259

Open p-wysocki opened 4 months ago

p-wysocki commented 4 months ago

Context

This is an effort to increase Large Language Models tests coverage in OpenVINO GenAI.

Working on this task will let you familiarize yourself with:

If you would like to add a new model which there's not a task for, please let us know! We would love to get outside ideas.

What needs to be done?

  1. Try running an LLM specified in your specific task with every sample form https://github.com/openvinotoolkit/openvino.genai/tree/master/samples.
  2. If it doesn't fail and outputs a reasonable responce, the PR should be submitted to extend the supported models list.
  3. If the model is small enough to fit into the default GitHub Actions runner, add the model to tests here.
  4. If the comparison against python transformers fails it's worth adding a test for the model anyway, just without comparison against python, leave a comment in workflow code if this is the case. Since default runners are available for everyone, one can verify the test passes by opening a PR in their fork first.

Example Pull Requests

Example commit: https://github.com/openvinotoolkit/openvino.genai/commit/bf4c20010bd26733b1603e38a0dd894f683de077#diff-2c8a6fc2893aa2e1103985c1ee763cc325de6042ea66a11ae30428d77e73e416

Resources

List of tasks in the effort

Expand me - https://github.com/openvinotoolkit/openvino.genai/issues/260 - https://github.com/openvinotoolkit/openvino.genai/issues/261 - https://github.com/openvinotoolkit/openvino.genai/issues/262 - https://github.com/openvinotoolkit/openvino.genai/issues/263 - https://github.com/openvinotoolkit/openvino.genai/issues/264 - https://github.com/openvinotoolkit/openvino.genai/issues/265 - https://github.com/openvinotoolkit/openvino.genai/issues/266 - https://github.com/openvinotoolkit/openvino.genai/issues/267 - https://github.com/openvinotoolkit/openvino.genai/issues/268 - https://github.com/openvinotoolkit/openvino.genai/issues/269 - https://github.com/openvinotoolkit/openvino.genai/issues/270 - https://github.com/openvinotoolkit/openvino.genai/issues/271 - https://github.com/openvinotoolkit/openvino.genai/issues/272 - https://github.com/openvinotoolkit/openvino.genai/issues/273 - https://github.com/openvinotoolkit/openvino.genai/issues/274 - https://github.com/openvinotoolkit/openvino.genai/issues/275 - https://github.com/openvinotoolkit/openvino.genai/issues/276

Contact points

@Wovchena

Ticket

134074

github-actions[bot] commented 2 months ago

Thank you for looking into this issue! Please let us know if you have any questions or require any help.

Jessielovecodings commented 1 month ago

WLB

.take

github-actions[bot] commented 1 month ago

Thank you for looking into this issue! Please let us know if you have any questions or require any help.

kuanxian1 commented 3 weeks ago

Most of the files linked here are not found, would be great if anyone could update it, thanks!

p-wysocki commented 3 weeks ago

Hello @kuanxian1, thanks for pointing our attention to it. @Wovchena I think the links are on openvino_notebooks repository, do you know who we could ask for an update?

Wovchena commented 3 weeks ago

I updated this issue description.