Frame-by-Frame Annotation Export Between Keyframes on Video Object Tracking

[dnnxl]

Hi! I would like to inquire whether it's possible to export frame-by-frame annotations between two keyframes using interpolation. For instance, if I have an object in keyframe 1 and keyframe 5, can I extract the individual annotations for the object in keyframes 2, 3, and 4 from the video object tracking export?

[makseq]

Yes, it is possible to export frame-by-frame annotations between two keyframes using interpolation in Label Studio. When you have annotated an object at keyframe 1 and keyframe 5, Label Studio can interpolate the annotations for frames 2, 3, and 4, and include these interpolated annotations in the export.

To achieve this, you need to enable keyframe interpolation during the export process. This can be done by setting the interpolate_key_frames parameter to True when creating an export snapshot. This tells Label Studio to generate annotations for all frames between your keyframes based on interpolation.

Here’s how you can do it using the Label Studio SDK:

from label_studio_sdk import Client

# Connect to Label Studio
ls = Client(url='http://localhost:8080', api_key='YOUR_API_KEY')

# Get your project
project = ls.get_project(PROJECT_ID)

# Create an export snapshot with keyframe interpolation enabled
export_result = project.export_snapshot_create(
    title='Export with Interpolated Keyframes',
    interpolate_key_frames=True
)

# Get the export ID
export_id = export_result['id']

# Wait for the export to complete (you may need to implement a waiting mechanism)

# Download the export
status, filename = project.export_snapshot_download(
    export_id, export_type='JSON', path='.'
)

By setting interpolate_key_frames=True, the exported annotations will include all interpolated frames between your keyframes. This means that the annotations for frames 2, 3, and 4 will be included in your export, based on the interpolation between the annotations at keyframes 1 and 5.

Note: Make sure that your Label Studio version supports keyframe interpolation during export. This feature is available in Label Studio versions that include video object tracking capabilities.

Links Used:

Label Studio SDK - Project Class

[tangfucius]

Thanks @makseq, the SDK solution works. Is this supported in API, and are there plans to add this option to the UI as well?