Potential Bias in Dataset and Model

yeyimilk / LLMGeo

LLMGeo: Benchmarking Large Language Models on Image Geolocation In-the-wild

0 stars 0 forks source link

Potential Bias in Dataset and Model #1

Closed yunusserhat closed 3 days ago

yunusserhat commented 4 days ago

Hi,

I noticed that both the dataset used for benchmarking (Google Street View) and the model with the best performance in the benchmark (Gemini) belong to Google. Could this impact the model's performance and bias?

Thanks

yeyimilk commented 3 days ago

Hi @yunusserhat you are right. We are not sure if Geimini had been trained on Google Street View data. We mentioned this concerned in the paper