Revert change to "Finish the job" behavior on app.cvat.com

[ChristianIngwersen]

Actions before raising this issue

• [X] I searched the existing issues and did not find anything similar.
• [X] I read/searched the docs

Is your feature request related to a problem? Please describe.

When finishing a job it would previously change the job stage to acceptance instead of annotation. Today when you press "Finish the job" it will change the state to completed instead.

There is however still a dropdown where you can change the job state to e.g. completed, but now you will need a lot of clicks to change the stage of a video.

Describe the solution you'd like

|t would be nice to either revert how this behaves or to add a drop-down menu that allows you to easily change the stage of a job.

Describe alternatives you've considered

Drop-down menu to easily change the job stage

Additional context

I'm using the web interface app.cvat.com

[zhiltsov-max]

Hi, could you describe your use case better? We'd like to know why this specific state (acceptance, completed) is important for your annotation pipeline, and why the updated version is worse.

[ChristianIngwersen]

Hi @zhiltsov-max ,

Thanks for taking the time! Our current workflow is the following:

1. Supervisor adds a new job and assigns to an annotator (annotation new)
2. Annotator annotates job and changes state to completed (annotation new -> in progress -> completed)
3. Supervisor checks all jobs in annotation completed and if it needs correction moves it to validation new and else to acceptance completed

So the idea is to make it easier for the supervisor to change it to acceptance. The argument for this would be that our supervisor generally spends less time per job and thus it would be nice to minimize the clicks needed to change the stage.

However, I see that it makes sense that once someone presses finish is moves the state to completed within a given stage. So maybe a better solution would be a dropdown within the job to change the stage of the job and not only the state?

Again thanks for the great work!

[zhiltsov-max]

Ok, so if I understand correctly, the problem is that if the supervisor / reviewer opens a job, they can't advance the job from the review stage to the acceptance directly, and instead they need to move back to the tasks page and do it there? In this case, I think, a temporary solution for you could be that the reviewer opens the job for review in a new tab, reviews the job, and then closes the opened tab and switches the stage on the task page. But it's an interesting use case, which we need to think about it. Thank you for sharing this information with us.

@bsekachev

[ChristianIngwersen]

Exactly!

And yes that has also been the new hopefully temporary solution :-)

[Prehani]

+1 for this; we use job stage + state to know when to pull labels

in our previous use case, when using the "Finish the Job" function, it would set the stage to "acceptance" and the state to "completed"; now, it maintains the same stage (usually "validation"), and only updates the state--when (I manually switch the stage, it resets the state

for my use case, we run the CVAT server on a GCP VM with a Google CloudSQL postgres instance; I use bigquery to scan for new jobs nightly that have a correct "assigned to" status, an accepted stage, and a completed state; if all 3 are met, it then adds them into our data lake

[bsekachev]

"assigned to" status, an accepted stage, and a completed state

why not modify condition on your side? I do not see in your pipeline why stage is important at all.

I manually switch the stage, it resets the state

This is expected. When stage changed, state is reset, e.g. when job goes from annotation to validation, another person (responsible for validation process, usually task owner or assignee) has to see that he has a new job to review. "validation", "new" are indicators for that.

[Prehani]

thanks for the prompt reply:

why not modify condition on your side? I do not see in your pipeline why stage is important at all.

we use the various stages for internal tracking of where things are; a few examples of common combinations we use:

1. stage annotation/state new are frames awaiting labeling
2. stage annotation/state in progress are frames...in progress
3. stage annotation/state completed are frames ready for initial QA
4. stage validation/state new are frames that we prepopulate with labels that solely need QA
5. stage validation/state accepted are frames that have made it past the initial cycle of QA
6. stage validation/state rejected are frames that I have rejected back to QA
7. stage acceptance/state completed are frames as I mentioned that are fully validated and ready only using state is not feasible; only using stage opens the flood gates for errant labels to be submitted

the usage of the various conditions on my end are fail safes; it is better to not have labels enter our lake because a member of the labeling team (or I) accidentally set it to accepted. the accepted/completed combination + assigned to field ensures with no uncertainty that the frames are fully reviewed. as I'm sure you can imagine it becomes a bigger headache to back track and delete the frames in the event that they are submitted as complete before being fully complete

This is expected. When stage changed, state is reset, e.g. when job goes from annotation to validation, another person (responsible for validation process, usually task owner or assignee) has to see that he has a new job to review. "validation", "new" are indicators for that

definitely, this makes total sense--my issue is more that previously I was able to click 1 button and have it automatically set the correct stage and state whereas now, when I fix the stage, the state resets. I understand why the state resets, but my issue is now I have to manually change it. this is a minor inconvenience, but when it used to just be a single button click, it's a bit of a pain to have to do manually

our process is centered around reducing pain points surrounding data labeling/QA/loading. we use voxel fiftyone for instance to send jobs automatically on a weekly basis and properly set up the job stage/state; in this case, we use the various values for state/stage/assigned to as layers of checks to make sure that everything is in order before uploading

if I am missing something/if there is a better way to do this, please let me know. ideally, if I could just create macros to create the job state, that would be optimal