search

cvat-ai / cvat

Annotate better with CVAT, the industry-leading data engine for machine learning. Used and trusted by teams at any scale, for data of any scale.
https://cvat.ai
MIT License
12.45k stars 2.99k forks source link

Using CVAT Event Log CSV to calculate Time taken for Annotation for each Image in a Job #6593

Open priyanshum-cashify opened 1 year ago

priyanshum-cashify commented 1 year ago

My actions before raising this issue

I am trying to calculate the exact time Taken for Annotation in each Image in a job.

As far as I understand, there are two approaches to calculating the Annotation Time for each Image: Method 1. Simply calculate the Time Taken in completing the entire Job and then get an average Annotation time by dividing by the number of Images in the Job. The method is quick but does not tell the exact time Taken in each image.

Method 2: To find and group the activities which belong to a particular Frame No and then add all the working time in the group to calculate the Time spent on Annotation for that particular Image.

I am attempting Method 2.

While attempting the same, I have come across a few Bugs which are as follows:

  1. Whenever one edits any Uploaded Annotation then, the corresponding Client Activity is NOT SHOWN in the Events Log CSV, only when we save the Job, a sever activity 'update:shapes' is shown, so one is not able to find frames Annotation Edit working time.
  2. The 'load:job' scope (i.e activity) does not tell that on which Frame number the Job is loaded, as the Job can be loaded in 2 ways, 1st by Clicking on the Job number in the Task page(this loads the first Frame) and 2nd the can also be loaded by Frame URL directly(this can load any particular Frame in the Job), so one is unable to decide that on which Frame exactly a Job was Loaded by the User

Please share your thoughts on the above findings, it would be a great help in exactly calculating the Time taken for Annotation per image if the above point(s) are handled.

my CVAT version- 2.4.4

amit-cashify commented 1 year ago

@bsekachev Any thoughts on this? Are we missing something very basic? We are happy to contribute an API for annotation/reviewing time taken per image/frame if we are able to understand and solve this.

amit-cashify commented 1 year ago

@bsekachev Any thoughts on this? Are we missing something very basic? We are happy to contribute an API for annotation/reviewing time taken per image/frame if we are able to understand and solve this.

@nmanovic Any help would be much appreciated.

priyanshum-cashify commented 1 year ago

Any Insight will be much appreciated. @bsekachev

nikste commented 11 months ago

is there any update on this ?

bsekachev commented 8 months ago

I may recommend to consider change:frame events only and their timestamps. If nothing happened on a frame, like user immediately did 1 -> 2 -> 3, it will show change:frame event from 1 to 3. Time between next change:frame event and previous change:frame event should approximately be time spend on a frame.

You may also substract change:frame duration (while data are fetching from the server and decoding)