search

uptrain-ai / uptrain

UpTrain is an open-source unified platform to evaluate and improve Generative AI applications. We provide grades for 20+ preconfigured checks (covering language, code, embedding use-cases), perform root cause analysis on failure cases and give insights on how to resolve them.
https://uptrain.ai/
Apache License 2.0
2.1k stars 174 forks source link

No warning when creating a New Project with the same name #602

Open cfrenchi opened 4 months ago

cfrenchi commented 4 months ago

Describe the bug I created an evaluation project called "Test". I then created another project called "Test" and the resulting scores are all added to the one project "Test". Is this expected behavior? Different .jsonl files can be used for the different New Projects. Clicking New Project implies a new unique project.

This may or may not be a bug depending on design plans but the wording of New Project implies something will be created not added to/joined if already created.

To Reproduce Steps to reproduce the behavior:

  1. Successfully create a New Project called "Test"
  2. See that evaluations were created
  3. Navigate to Home
  4. Successfully create another New Project called "Test"
  5. See that the evaluations have been added to the first project called "Test

Should there be a warning that the project exists?

Expected behavior I expected to get a warning for the project name already being used. When creating a New Project with the same name, should these be joined?

Screenshots

image

Created the New Project "Test" using this jsonl: {"question": "Which is the most popular global sport?", "context": "The popularity of sports can be measured in various ways, including TV viewership, social media presence, number of participants, and economic impact. Football is undoubtedly the world's most popular sport with major events like the FIFA World Cup and sports personalities like Ronaldo and Messi, drawing a followership of more than 4 billion people.", "response": "Football is the most popular sport with around 4 billion followers worldwide"}

image

Created the second New Project "Test" using this jsonl: {"question": "Who created the Python language?", "context": "Python, created by Guido van Rossum in the late 1980s, is a high-level general-purpose programming language. Its design philosophy emphasizes code readability, and its language constructs aim to help programmers write clear, logical code for both small and large-scale software projects.", "response": "Python language was created by Guido van Rossum."}

Additional context I also have a project "test" that was not joined which was good to see and expected.

cfrenchi commented 4 months ago

I would argue that New Project should create a unique project. If we want to add evaluations to an existing project that should be done through the Evaluations page which isn't a current option. The fact that I would need to create a New Project with the same name as an existing project to update an Evaluation of an already created project feels strange.

ashish-1600 commented 4 months ago

Thanks @cfrenchi . We are working on this. We will let you know once it is done from our side.

ashish-1600 commented 2 months ago

Hey @cfrenchi . We have resolved this issue. Can you please confirm it once? We have pushed new images for the uptrain-dashboard and uptrain-backend-server.

cfrenchi commented 2 months ago

@ashish-1600 with the changes, and this being more of a ux/design issue, what is the new expected user flow? Is there a specific commit or documentation regarding the changes?