search

openvinotoolkit / openvino.genai

Run Generative AI models using native OpenVINO C++ API
Apache License 2.0
76 stars 115 forks source link

[Good First Issue]: Verify dolly-v2-3b with GenAI text_generation #275

Open p-wysocki opened 4 months ago

p-wysocki commented 4 months ago

Context

This task regards enabling tests for dolly-v2-3b. You can find more details under openvino_notebooks LLM question answering README.md.

Please ask general questions in the main issue at https://github.com/openvinotoolkit/openvino.genai/issues/259

What needs to be done?

Described in the main Discussion issue at: https://github.com/openvinotoolkit/openvino.genai/issues/259

Example Pull Requests

Described in the main Discussion issue at: https://github.com/openvinotoolkit/openvino.genai/issues/259

Resources

Contact points

Described in the main Discussion issue at: https://github.com/openvinotoolkit/openvino.genai/issues/259

Ticket

No response

Vishwa44 commented 4 months ago

.take

Vishwa44 commented 4 months ago

Hi @Wovchena can this task be assigned to me

p-wysocki commented 4 months ago

Hello @Vishwa44, we enabled the take command in GenAI repo just after you wrote it. :)

I assigned you the task, thanks for taking a look! Please let us know if you have any questions.

Vishwa44 commented 4 months ago

@p-wysocki I just got done testing the model. It's working as expected I'll raise a pr adding it to the supported model list

p-wysocki commented 4 months ago

Perfect! Please make sure to link it to the issue. :)

Vishwa44 commented 4 months ago

Hi @p-wysocki, this is the link to my pr verifed-dolly

p-wysocki commented 4 months ago

Hello @Vishwa44, thank you for your contribution! Could you please add tests? You can check other completed issues for reference or the issue description at https://github.com/openvinotoolkit/openvino.genai/issues/259.

Vishwa44 commented 4 months ago

Hi @p-wysocki I'm really sorry but I'm new to this, can you share some handy guide on how to add tests for github actions runner, I'll make the required changes and get it done asap

mlukasze commented 4 months ago

@Wovchena || @pavel-esir : could you please close a ticket, both PRs has been merged. Also, please add 2024.1 as a milestone to this ticket. Thanks.

pavel-esir commented 3 months ago

@mlukasze those prs were for Readme changes, but there is a left test for dolly 2-3b. Let's close after the tests are implemented. @Vishwa44 sorry for late reply, you can take as example already existing test where we compare output generated by us with HF transformers output, e.g.: https://github.com/openvinotoolkit/openvino.genai/blob/master/.github/workflows/causal_lm_cpp.yml#L151-L168

you can simplify code above and compare only for greedy scenario, run greedy_causal_lm instead of beam_search_causal_lm and generate in HF with much shorter command generate(**tokenized, max_length=100, do_sample=False)

Please let me know if you have any questions

mlukasze commented 3 months ago

got it, thanks for clarification :)

Vishwa44 commented 3 months ago

@pavel-esir I'll raise the pr asap for adding the test, thank you

andrei-kochin commented 2 months ago

@Vishwa44 are you still working on this? there is more than month without an update

p-wysocki commented 2 months ago

The model still needs to be validated and the task is now open again. @Vishwa44 if you're still working on this please let us know, for now I reopened the task.

Vishwa44 commented 2 months ago

@p-wysocki sorry for the late reply, I'll resume on this and finish this task

Apoorv012 commented 1 month ago

@p-wysocki Is this issue up for grab?

Apoorv012 commented 1 month ago

.take

github-actions[bot] commented 1 month ago

Thank you for looking into this issue! Please let us know if you have any questions or require any help.