HumanSignal / label-studio

Label Studio is a multi-type data labeling and annotation tool with standardized output format
https://labelstud.io
Apache License 2.0
18.39k stars 2.31k forks source link

Filters applied twice in combination with bad transaction isolation lead to exponential growth of filters. #1394

Closed karelserruys-foodpairing closed 2 weeks ago

karelserruys-foodpairing commented 3 years ago

Describe the bug We're running Label Studio 1.0.2 with PostgreSQL 13 as database. When refreshing a project view, the frontend makes calls to the backend to reapply the all filters which are present on the view. Additionally, the API is called once for each filter even though when using multiple filters, they need to be applied using a single API call. Postgres's default transactional isolation is read committed. This in combination with transactions that add filters at almost the same moment, results in an exponential growth of filters eventually leading to an irresponsive application (because the amount of filters is to large to fetch from the backend). When settings our Postgres to repeatable read partially solves the issue. Filters are no longer growing exponentially, because one of the simultaneous transactions fails. The problem is that the frontend keeps retrying both API calls until the moment the both succeed. This sometimes takes no more than a second, but sometimes it takes up to 30 seconds.

To Reproduce Steps to reproduce the behavior:

  1. Go to a project
  2. Add a view
  3. Add at least 2 filters
  4. Refresh & watch the amount of calls that are being done to the backend adding filters

Expected behavior Filters should only be added once.

Screenshots Filters on the project view image

Calls adding filters after first refresh image

Refresh again, resulting filters are: image

Calls when refreshing one more time: image

Running locally with Postgres transaction isolation set to repeatable read image

Environment (please complete the following information):

Additional context Add any other context about the problem here.

makseq commented 3 years ago

@karelserruys-foodpairing Thank you for your report and ideas. We are working on this issue and we've decided to redesign all filter logic: we want to have an Apply button for filters and it solves this problem. Hope, it will be released soon.

remoyson commented 2 years ago

@makseq Any updates on this issue? Having the exact same problem with filters being applied multiple times and blocking application loading. Looking forward to a fix of this problem :)

makseq commented 2 years ago

@remoyson We have some fixes in the latest master, but not all. Could you check the latest commits?

karelserruys-foodpairing commented 2 years ago

@remoyson @makseq For me the issue was fixed when I upgraded to version 1.3.0. I think the issue can be closed? Regards Karel