My Herodotus
Herodotus, often considered the "Father of Historians," was also a prolific travel journelist. His Histories act as one of the very first travel guides of the Ancient Mediterranean.
This project demonstrates how to create your own travel guide using generative AI hosted on Google Cloud--in effect, your very own Herodotus.
Architecture
This project uses a three-tier architecture, with a simple web frontend, an application tier, and a data tier.
This project uses the following Google services:
This project also uses the following libraries:
Model evaluations
This system allows the usage of three related LLM models:
- The out-of-the-box Gemini 1.5 Flash model
- A tuned version of the Gemini 1.5 Flash model, trained on the Guanaco dataset.
- A Gemma 2 open source model. This model currently cannot be evaluated with the Evaluations API.
These models have been evaluated against the following set of metrics.
The following table shows the evaluation scores for each of these models.
| Model | ROUGE | Closed domain | Open domain | Groundedness | Coherence | Date of eval |
|---|---|---|---|---|---|---|
| Gemini 1.5 Flash | 1.0[1] | 0.52 | 1.0 | 1.0[1] | 3.8 | 2024-11-07 |
| Tuned Gemini | 0.41 | 0.8 | 1.0 | 0.6 | 3.8 | 2024-11-07 |
[1]: Gemini 1.5 Flash responses were used as the ground truth for all other models.