redhat-et / foundation-models-for-documentation

Improve ROSA customer experience (and customer retention) by leveraging foundation models to do “gpt-chat” style search of Red Hat customer documentation assets.
Other
24 stars 11 forks source link

metrics evalution nb for demo #41

Closed suppathak closed 11 months ago

suppathak commented 1 year ago

24

review-notebook-app[bot] commented 1 year ago

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

suppathak commented 1 year ago

Thanks for the review @Shreyanand . I will address them in the next commit.

codificat commented 1 year ago

One thing we discussed earlier today was the option of adding a "human review" metric. Do we want to do that in this notebook before merging? Do you need collaboration for that?

Besides that, I would suggest editing the README in the notebooks and add an entry in the list of notebooks for this one.

There's also a merge conflict due to an updated Pipfile{.lock} in master

suppathak commented 1 year ago

One thing we discussed earlier today was the option of adding a "human review" metric. Do we want to do that in this notebook before merging? Do you need collaboration for that?

Thanks @codificat . I would like to add "human evaluation metric" in this notebook. With this addition (which can act as a baseline metric), i think it would be easier to interpret other metrics. Having a collaboration for it is great. But first I think, we need to create some basic rules for evaluation. I am now trying to come up with some set of rules, which we can discuss in our sync-ups. Let me know wyt?

Besides that, I would suggest editing the README in the notebooks and add an entry in the list of notebooks for this one.

Sure!

There's also a merge conflict due to an updated Pipfile{.lock} in master

ok!! Thanks @codificat .

Shreyanand commented 1 year ago

@suppathak What is pending here?

suppathak commented 1 year ago

@suppathak What is pending here?

All good. Feel free to review it. Thank you @Shreyanand .