search

GoogleCloudPlatform / applied-ai-engineering-samples

This repository compiles code samples and notebooks demonstrating how to use Generative AI on Google Cloud Vertex AI.
https://googlecloudplatform.github.io/applied-ai-engineering-samples/
Apache License 2.0
357 stars 129 forks source link

Adding a Tendency Based Evaluation Notebook #109

Open alejandroballestae opened 1 month ago

alejandroballestae commented 1 month ago

This notebook explores the application of statistical/tendency methods to gain insights into the behavior of the Vertex AI Python SDK for Rapid Evaluation for generated answers during Question Answering (QA) tasks. Due to the inherent variability, non-deterministic or probabilistic outcomes in these tasks, relying on a single execution may not provide a comprehensive understanding of the real performance.

review-notebook-app[bot] commented 1 month ago

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

review-notebook-app[bot] commented 3 weeks ago

View / edit / reply to this conversation on ReviewNB

RajeshThallam commented on 2024-10-14T14:20:48Z ----------------------------------------------------------------

Add links to open the notebook in Colab, Workbench, Colab Enterprise and Github. Follow this notebook for example.


review-notebook-app[bot] commented 3 weeks ago

View / edit / reply to this conversation on ReviewNB

RajeshThallam commented on 2024-10-14T14:20:49Z ----------------------------------------------------------------

Can you test with 3.10 or above? Most env are now upgraded to 3.10.


review-notebook-app[bot] commented 3 weeks ago

View / edit / reply to this conversation on ReviewNB

RajeshThallam commented on 2024-10-14T14:20:50Z ----------------------------------------------------------------

We are moving away from "Rapid Evaluation" branding and using "Gen AI Evaluation Service". Please update accordingly.

Please add references to product documentation where appropriate.


review-notebook-app[bot] commented 3 weeks ago

View / edit / reply to this conversation on ReviewNB

RajeshThallam commented on 2024-10-14T14:20:50Z ----------------------------------------------------------------

Please update module name with new GA version: google-cloud-aiplatform[evaluation]

Test with 1.70.0 version of Vertex AI SDK


review-notebook-app[bot] commented 3 weeks ago

View / edit / reply to this conversation on ReviewNB

RajeshThallam commented on 2024-10-14T14:20:51Z ----------------------------------------------------------------

remove references to project ids. Please use [your-project-id] ot something similar.


review-notebook-app[bot] commented 3 weeks ago

View / edit / reply to this conversation on ReviewNB

RajeshThallam commented on 2024-10-14T14:20:52Z ----------------------------------------------------------------

typo contex --> context

The notebook reference does not exist anymore after migration to GA version. Please update accordingly.


  • © Githubissues.
  • Githubissues is a development platform for aggregating issues.