hkust-nlp / AgentBoard

An Analytical Evaluation Board of Multi-turn LLM Agents
250 stars 26 forks source link

Moviedb needs commercial license #24

Closed kesimeg closed 1 month ago

kesimeg commented 1 month ago

Hello, the license agreement of Moviedb states that even using moviedb API for validating a machine learning system such as an LLM is considered commercial use. In the list of examples what they consider commercial use includes the following statement:

"Training or validating a machine learning or artificial intelligence system (including large language models and Chatbots) using TMDB content, collecting data sets containing TMDB Content for such training or validation, or providing archived or cached data sets containing TMDB Content to another person or entity for such training or validation." .

So from this statement (and the rest of the license) it seems that even without making any profit (such as research projects) using the API for validation is commercial use. I think most users of this dataset are probably researchers. So it would be better if a benchmark dataset like this can be used without commercial use agreement which will make it highly accessible. Is it possible for you to replace the API with another open source alternative? Maybe you can create a similar architecture to "academia" API where the API is offline.

yc1999 commented 1 month ago

Hello, we contacted TMDB to inquire about commercial use, and we received the following response:

image

Researchers are allowed to use the TMDB API as long as it’s not for profit. However, if researchers use our benchmark for developing commercial products, it is recommended that they obtain a commercial agreement.